We may earn an affiliate commission when you visit our partners.

Serverless Data Processing with Dataflow

Operations em Português Brasileiro

Google Cloud Training

Na última parte da série de cursos do Dataflow, vamos abordar os componentes do modelo operacional do Dataflow. Veremos ferramentas e técnicas para solucionar problemas e otimizar o desempenho do pipeline. Depois analisaremos as práticas recomendadas de teste, implantação e confiabilidade para pipelines do Dataflow. Por fim, faremos uma revisão dos modelos, que facilitam o escalonamento dos pipelines do Dataflow para organizações com centenas de usuários. Essas lições garantem que a plataforma de dados seja estável e resiliente a circunstâncias imprevistas.

Enroll now

Or subscribe to Coursera Plus

And get unlimited access to Coursera

Here's a deal for you

Save money when you learn with a deal that may be relevant to this course.

All coupon codes, vouchers, and discounts are applied automatically unless otherwise noted.

Valid until August 30

Google AI App Builder

Learn how to use Gemini API and API Studio with a three-course series from Google DeepMind

What's inside

Syllabus

Introduction

Este módulo mostra o resumo do curso

Monitoring

Neste módulo, aprendemos a usar a página Lista de jobs para filtrar os jobs que queremos monitorar ou investigar. Vimos como as guias Gráfico de job, Informações do job e Métricas do job mostram um resumo abrangente do job do Dataflow. Por fim, aprendemos a usar a integração do Dataflow com o Metrics Explorer e criar políticas de alertas para as métricas do Dataflow.

Traffic lights

Read about what's good

what should give you pause

and possible dealbreakers

Analisa problemas e soluções comuns para criar pipelines de dados confiáveis e com desempenho otimizado

Descreve técnicas para monitorar e solucionar problemas no Google Dataflow

Aborda modelos Flex, permitindo a padronização e reutilização de código de pipeline do Dataflow

Indicado para profissionais com experiência em desenvolvimento de pipelines do Dataflow

Oferece práticas recomendadas de teste, implantação e confiabilidade para pipelines do Dataflow

Save this course

Create your own learning path. Save this course to your list so you can find it easily later.

Save

Reviews summary

Operações dataflow essenciais e práticas

Segundo os alunos, este curso de Google Cloud Dataflow é altamente prático e focado nas operações essenciais da plataforma. Muitos destacam os laboratórios e as demonstrações como pontos fortes, por serem claros e úteis para a fixação do conteúdo e aplicação em cenários reais. A abordagem sobre monitoramento, depuração e solução de problemas é frequentemente elogiada, assim como a introdução aos modelos Flex, considerados um recurso valioso. Embora a maioria o considere relevante para profissionais, alguns alertam que pode exigir conhecimento prévio em GCP e Apache Beam, ou ser superficial para usuários muito avançados. O instrutor é geralmente bem avaliado por sua didática. O curso parece manter sua relevância com o tempo.

Cobertura detalhada e relevante dos modelos Flex.

"Este é um bom curso para entender a parte operacional do Dataflow, especialmente os templates Flex, que são muito úteis."

"Sensacional! Este curso preenche uma lacuna importante sobre as operações do Dataflow. Os modelos Flex são um divisor de águas."

"A explicação sobre os modelos Flex foi excelente e me mostrou como padronizar e reutilizar o código de pipeline."

Atividades práticas e demonstrações são claras e eficazes.

"Os laboratórios são muito práticos e ajudam a fixar o conteúdo. Pude aplicar o que aprendi imediatamente."

"O curso é muito bem estruturado e focado na prática. Os demos são claros e fáceis de acompanhar."

"Os laboratórios são a melhor parte para mim, realmente me ajudaram a entender o fluxo de trabalho na prática."

Aborda ferramentas e técnicas para operações de Dataflow.

"O curso é excelente, aborda tudo sobre operações de Dataflow. Os laboratórios são muito práticos e ajudam a fixar o conteúdo."

"Este conteúdo é super relevante para engenheiros de dados que trabalham com Dataflow. A parte de troubleshooting é muito útil."

"Aprendi bastante sobre monitoramento e depuração. As demonstrações são claras e diretas ao ponto, o que me ajudou muito."

Alguns tópicos podem ser rasos para usuários com experiência.

"Achei o curso um pouco superficial em alguns pontos, especialmente na otimização de desempenho. Para quem já tem experiência, pode ser básico demais."

"Gostaria de mais exemplos de casos reais no curso, senti falta de aprofundar em certas situações."

"Útil, mas senti falta de mais detalhes sobre a integração com outras ferramentas do GCP, o que seria interessante para mim."

Pode ser desafiador para iniciantes sem experiência prévia.

"Achei muito difícil para quem não tem background na plataforma. Os pré-requisitos não são claros e abandonei na metade."

"Não senti que fosse para iniciantes totais; uma base em GCP é bem-vinda para aproveitar melhor o conteúdo."

"Para quem já tem alguma base em GCP, o curso é excelente, mas sem isso, pode ser um desafio."

Activities

Be better prepared before your course. Deepen your understanding during and after it. Supplement your coursework and achieve mastery of the topics covered in Serverless Data Processing with Dataflow: Operations em Português Brasileiro with these activities:

Revise Apache Beam Pipeline Components

Show steps

Review the concepts of Apache Beam pipelines. It will help you comprehend the operational components better.

Browse courses on Apache Beam

Show steps

Go through the documentation of Apache Beam Pipeline Components.
Refer to tutorials or online resources to understand their practical usage.

Job List and Monitoring Exercises

Show steps

Practicing job list filtering and monitoring will enhance your ability to manage and debug pipelines.

Show steps

Use the Dataflow web interface to access the Job List.
Filter jobs based on various parameters (e.g., status, start time).
Explore the Job Graph, Job Details, and Job Metrics sections.
Create and manage job alerts to track job status changes.

Error Reporting and Logging Analysis

Show steps

Understanding error reporting and logging will equip you with skills to troubleshoot and debug pipelines effectively.

Browse courses on Error Reporting

Show steps

Follow tutorials on error reporting and logging in Dataflow.
Explore the Error Reporting page in the Dataflow console.
Analyze log files generated by your pipelines.
Use the Metrics Explorer to identify potential errors.

Two other activities

Expand to see all activities and additional details

Show all five activities

Dataflow CI/CD Pipeline Development

Show steps

By creating a CI/CD pipeline for your Dataflow pipelines, you can ensure their reliability and maintainability.

Browse courses on CI/CD Pipeline

Show steps

Set up a CI/CD tool (e.g., Jenkins, Azure DevOps).
Define the pipeline stages (e.g., unit tests, integration tests, deployment).
Integrate your Dataflow pipelines into the CI/CD process.
Configure automated triggers to initiate the pipeline.
Monitor the pipeline execution and receive notifications.

Dataflow Flex Templates Study Group

Show steps

Engage in discussions with peers about the advantages and use cases of Dataflow Flex Templates.

Show steps

Join online forums or discussion groups related to Dataflow Flex Templates.
Share experiences and insights with other Dataflow users.
Collaborate on sample templates and explore different use cases.
Present and discuss findings within the study group.

Career center

Learners who complete Serverless Data Processing with Dataflow: Operations em Português Brasileiro will develop knowledge and skills that may be useful to these careers:

Data Engineer

Data Engineers are responsible for the design and implementation of data pipelines. They build and maintain the infrastructure that moves data around an organization. This course may be useful to Data Engineers who want to learn how to use Apache Beam and Google Cloud Dataflow to build and manage data pipelines.

See salaries and explore the career path for Data Engineer

Cloud Architect

Cloud Architects design, build, and maintain cloud computing solutions. They work with customers to understand their business needs and then design and implement cloud solutions that meet those needs. A course in Apache Beam and Google Cloud Dataflow can be useful for Cloud Architects who want to learn how to use these technologies to build and manage data pipelines.

See salaries and explore the career path for Cloud Architect

Data Scientist

Data Scientists use data to solve business problems. They collect, clean, and analyze data to identify trends and patterns. This course may be useful to Data Scientists who want to learn how to use Apache Beam and Google Cloud Dataflow to build and manage data pipelines.

See salaries and explore the career path for Data Scientist

Software Engineer

Software Engineers design, develop, and maintain software applications. They work with customers to understand their needs and then design and implement software solutions that meet those needs. A course in Apache Beam and Google Cloud Dataflow can be useful for Software Engineers who want to learn how to use these technologies to build and manage data pipelines.

See salaries and explore the career path for Software Engineer

Data Analyst

Data Analysts collect, clean, and analyze data to identify trends and patterns. They use this information to make recommendations to businesses on how to improve their operations. This course may be useful to Data Analysts who want to learn how to use Apache Beam and Google Cloud Dataflow to build and manage data pipelines.

See salaries and explore the career path for Data Analyst

Business Analyst

Business Analysts work with businesses to identify and solve business problems. They use data to understand the business and then develop and implement solutions that improve the business. A course in Apache Beam and Google Cloud Dataflow may be useful for Business Analysts who want to learn how to use these technologies to build and manage data pipelines.

See salaries and explore the career path for Business Analyst

Product Manager

Product Managers are responsible for the development and management of products. They work with customers to understand their needs and then design and implement products that meet those needs. A course in Apache Beam and Google Cloud Dataflow may be useful for Product Managers who want to learn how to use these technologies to build and manage data pipelines.

See salaries and explore the career path for Product Manager

Project Manager

Project Managers are responsible for the planning and execution of projects. They work with stakeholders to define the project scope, timeline, and budget. A course in Apache Beam and Google Cloud Dataflow may be useful for Project Managers who want to learn how to use these technologies to build and manage data pipelines.

See salaries and explore the career path for Project Manager

Database Administrator

Database Administrators are responsible for the maintenance and administration of databases. They work with databases to ensure that they are running smoothly and that data is safe and secure. A course in Apache Beam and Google Cloud Dataflow may be useful for Database Administrators who want to learn how to use these technologies to build and manage data pipelines.

See salaries and explore the career path for Database Administrator

Systems Administrator

Systems Administrators are responsible for the maintenance and administration of computer systems. They work with systems to ensure that they are running smoothly and that data is safe and secure. A course in Apache Beam and Google Cloud Dataflow may be useful for Systems Administrators who want to learn how to use these technologies to build and manage data pipelines.

See salaries and explore the career path for Systems Administrator

Network Administrator

Network Administrators are responsible for the maintenance and administration of computer networks. They work with networks to ensure that they are running smoothly and that data is safe and secure. A course in Apache Beam and Google Cloud Dataflow may be useful for Network Administrators who want to learn how to use these technologies to build and manage data pipelines.

See salaries and explore the career path for Network Administrator

Security Analyst

Security Analysts are responsible for the security of computer systems and networks. They work with systems and networks to identify and mitigate security risks. A course in Apache Beam and Google Cloud Dataflow may be useful for Security Analysts who want to learn how to use these technologies to build and manage data pipelines.

See salaries and explore the career path for Security Analyst

Data Governance Analyst

Data Governance Analysts are responsible for the development and implementation of data governance policies and procedures. They work with businesses to ensure that data is used in a compliant and ethical manner. A course in Apache Beam and Google Cloud Dataflow may be useful for Data Governance Analysts who want to learn how to use these technologies to build and manage data pipelines.

See salaries and explore the career path for Data Governance Analyst

Data Privacy Analyst

Data Privacy Analysts are responsible for the protection of data privacy. They work with businesses to ensure that data is collected, used, and stored in a compliant and ethical manner. A course in Apache Beam and Google Cloud Dataflow may be useful for Data Privacy Analysts who want to learn how to use these technologies to build and manage data pipelines.

See salaries and explore the career path for Data Privacy Analyst

Data Protection Officer

Data Protection Officers are responsible for the implementation and enforcement of data protection policies and procedures. They work with businesses to ensure that data is collected, used, and stored in a compliant and ethical manner. A course in Apache Beam and Google Cloud Dataflow may be useful for Data Protection Officers who want to learn how to use these technologies to build and manage data pipelines.

See salaries and explore the career path for Data Protection Officer

Reading list

We've selected six books that we think will supplement your learning. Use these to develop background knowledge, enrich your coursework, and gain a deeper understanding of the topics covered in Serverless Data Processing with Dataflow: Operations em Português Brasileiro.

Secret Colors

Save

Este livro explora os princípios de projetar e construir aplicativos que processam grandes quantidades de dados. Ele abrange tópicos como armazenamento de dados, processamento de dados em lote e de streaming e gerenciamento de estado.

R: Recipes for Analysis, Visualization and Machine...

Save

Este livro fornece uma introdução ao uso do Python para engenharia de dados. Ele aborda tópicos como manipulação de dados, processamento de dados de streaming e aprendizado de máquina.

(English) R Recipes for Analysis, Visualization and Machine...

Paperback

$$$

(English) R: Recipes for Analysis, Visualization and Machine...

Kindle Edition

$$$

Mastering Spark for Data Science

Save

Este livro aborda os princípios de projetar e construir soluções orientadas a dados. Ele complementa o curso fornecendo uma perspectiva de arquitetura e ajudando os alunos a entender como o Dataflow se encaixa em arquiteturas de dados mais amplas.

(English) Mastering Spark for Data Science

Paperback

$$$

(English) Mastering Spark for Data Science

Kindle Edition

Deep Learning with Python, Second Edition

Save

Este livro fornece uma introdução prática ao aprendizado profundo. Ele aborda tópicos como redes neurais, convoluções e redes neurais recorrentes.

(English) Deep Learning with Python, Second Edition

Paperback

(English) Deep Learning with Python, Second Edition

Kindle Edition

Data Mesh

Save

Este livro explora o conceito de data mesh, que é uma abordagem emergente para gerenciar e usar dados. Ele complementa o curso abordando o papel do Dataflow no ecossistema de data mesh e fornecendo uma visão mais estratégica.

Supporting Shrinkage

Save

Este livro aborda a análise de dados em tempo real, que é um aspecto importante do processamento de dados em larga escala. Ele complementa o curso fornecendo uma visão geral das tecnologias e técnicas usadas para análise de dados em tempo real, incluindo o Dataflow.

(English) Supporting Shrinkage: Better Planning and Decision...

Paperback

(English) Supporting Shrinkage: Better Planning and Decision...

Kindle Edition

Help others find this course page by sharing it with your friends and followers: