We may earn an affiliate commission when you visit our partners.
Tim Warner
Azure Data Engineers design and implement data solutions in the Microsoft Azure cloud. This course will teach you how to use Azure products to perform batch data processing operations.
Read more
Azure Data Engineers design and implement data solutions in the Microsoft Azure cloud. This course will teach you how to use Azure products to perform batch data processing operations.
Read more
Azure Data Engineers design and implement data solutions in the Microsoft Azure cloud. This course will teach you how to use Azure products to perform batch data processing operations.
Long-running batch data processing can be difficult to manage locally - why not use Microsoft Azure? In this course, Building Batch Data Processing Solutions in Microsoft Azure, you’ll gain the ability to perform high-scale ETL and ELT operations entirely in the Azure public cloud. First, you’ll explore hosted Apache Spark processing with Azure Databricks. Next, you’ll discover how to transfer data in bulk with Azure Data Factory and Azure Data Explorer. Finally, you’ll learn how to handle stream data processing jobs with Azure Stream Analytics. When you’re finished with this course, you’ll have the skills and knowledge of batch data processing needed to earn your Azure Data Engineer certification and be productive with the Microsoft Azure Data Platform.
Register for this course and see more details by visiting:
OpenCourser.com/course/792n3w/building
What's inside
Syllabus
Course Overview
Develop a Batch Processing Solution with Azure Databricks
Develop a Batch Processing Solution with Azure Synapse Analytics
Develop a Batch Processing Solution with Azure Data Explorer
Read more
Syllabus
Course Overview
Develop a Batch Processing Solution with Azure Databricks
Develop a Batch Processing Solution with Azure Synapse Analytics
Develop a Batch Processing Solution with Azure Data Explorer
Read more
Implement Event Processing with Azure Stream Analytics
Good to know
Good to know
Know what's good ,
what to watch for , and
possible dealbreakers
Teaches skills, knowledge, and tools that are highly relevant to industry
Teaches skills, knowledge, and tools that are core for Azure Data Engineers
Develops skills for performing high-scale ETL and ELT operations in the Azure public cloud
Covers event processing with Azure Stream Analytics
Examines hosted Apache Spark processing with Azure Databricks
Explores data transfer in bulk with Azure Data Factory and Azure Data Explorer
Save this course
Save Building Batch Data Processing Solutions in Microsoft Azure to your list so you can find it easily later:
Save
Save
Activities
Be better prepared
before
your course. Deepen your understanding
during
and
after
it. Supplement your coursework and achieve mastery of the topics covered
in Building Batch Data Processing Solutions in Microsoft Azure with these
activities:
Review Azure Data Platform Architecture
Show steps
Refresh your knowledge of the Azure Data Platform's architecture and components, ensuring a strong foundation for understanding batch data processing solutions in Azure.
Show steps
-
Review Azure Storage services (Blob Storage, Data Lake Storage)
-
Understand Azure Compute services (Azure VMs, Azure Functions)
-
Explore Azure Data Services (Azure SQL Database, Azure Cosmos DB)
-
Learn about Azure Data Platform's integration and interoperability
-
Review Azure Data Platform security and governance features
Review Data Ingestion Techniques for Batch Data Processing
Show steps
Strengthen your foundation in data ingestion techniques by reviewing key concepts and practices relevant to batch data processing.
Browse courses on
Data Ingestion
Show steps
-
Understand different data sources and their characteristics
-
Explore data ingestion tools and technologies
-
Learn about data transformation and cleansing techniques
-
Practice data ingestion and preparation using sample datasets
Interactive Tutorial: Explore Azure Data Explorer for Batch Analytics
Show steps
Enhance your understanding of Azure Data Explorer's capabilities by following an interactive tutorial that demonstrates how to ingest, query, and visualize large-scale batch data.
Browse courses on
Azure Data Explorer
Show steps
-
Create an Azure Data Explorer cluster
-
Ingest data from various sources
-
Explore and analyze data using Kusto Query Language (KQL)
-
Visualize data using Azure Data Explorer's visualizations
-
Troubleshoot common issues
Five other activities
Expand to see all activities and additional details
Show all eight activities
Practice Azure Databricks for Spark Batch Processing
Show steps
Solidify your understanding of Spark Batch Processing concepts by working through a series of exercises that involve creating, running, and monitoring batch jobs in Azure Databricks.
Browse courses on
Azure Databricks
Show steps
-
Create an Azure Databricks workspace
-
Create a Spark DataFrame from a data source
-
Perform data transformations using Spark SQL
-
Write the transformed data to a data sink
-
Monitor the job execution using the Azure Databricks UI
Build an ETL Pipeline for Batch Data Processing
Show steps
Reinforce your understanding of Azure Data Factory's capabilities by creating an ETL pipeline that transforms and loads data from one system to another.
Browse courses on
Azure Data Factory
Show steps
-
Design the ETL pipeline architecture
-
Create a data source and sink
-
Transform the data using Data Factory transformations
-
Schedule and execute the pipeline
-
Monitor the pipeline execution
Hands-on Lab: Build a Batch Data Processing Solution with Azure Synapse Analytics
Show steps
Gain practical experience in building a batch data processing solution using Azure Synapse Analytics by following a hands-on lab that guides you through the entire process.
Browse courses on
Azure Synapse Analytics
Show steps
-
Create an Azure Synapse workspace
-
Ingest data from various sources
-
Transform and process data using Synapse SQL
-
Store the processed data in Azure Data Lake Storage
-
Monitor and troubleshoot the data processing pipeline
Contribute to the Apache Spark Community
Show steps
Deepen your understanding of Spark Batch Processing and contribute to the broader community by actively participating in the Apache Spark project.
Browse courses on
Apache Spark
Show steps
-
Identify areas within Apache Spark that align with your interests
-
Join the Apache Spark community and mailing lists
-
Review the Spark codebase and documentation
-
Contribute bug fixes or enhancements to the project
-
Engage in discussions and provide support to other community members
Develop a Batch Data Processing Solution for a Real-World Scenario
Show steps
Showcase your mastery of batch data processing by developing a comprehensive solution that addresses a specific business challenge or problem.
Show steps
-
Define the business requirements and objectives
-
Design the batch data processing architecture
-
Select and configure appropriate Azure services
-
Develop and implement the data processing pipeline
-
Monitor, evaluate, and optimize the solution
Review Azure Data Platform Architecture
Show steps
Refresh your knowledge of the Azure Data Platform's architecture and components, ensuring a strong foundation for understanding batch data processing solutions in Azure.
Show steps
Show steps
Show steps
- Review Azure Storage services (Blob Storage, Data Lake Storage)
- Understand Azure Compute services (Azure VMs, Azure Functions)
- Explore Azure Data Services (Azure SQL Database, Azure Cosmos DB)
- Learn about Azure Data Platform's integration and interoperability
- Review Azure Data Platform security and governance features
Review Data Ingestion Techniques for Batch Data Processing
Show steps
Strengthen your foundation in data ingestion techniques by reviewing key concepts and practices relevant to batch data processing.
Browse courses on
Data Ingestion
Show steps
Show steps
Show steps
- Understand different data sources and their characteristics
- Explore data ingestion tools and technologies
- Learn about data transformation and cleansing techniques
- Practice data ingestion and preparation using sample datasets
Interactive Tutorial: Explore Azure Data Explorer for Batch Analytics
Show steps
Enhance your understanding of Azure Data Explorer's capabilities by following an interactive tutorial that demonstrates how to ingest, query, and visualize large-scale batch data.
Browse courses on
Azure Data Explorer
Show steps
Show steps
Show steps
- Create an Azure Data Explorer cluster
- Ingest data from various sources
- Explore and analyze data using Kusto Query Language (KQL)
- Visualize data using Azure Data Explorer's visualizations
- Troubleshoot common issues
Five other activities
Expand to see all activities and additional details
Show all eight activities
Practice Azure Databricks for Spark Batch Processing
Show steps
Solidify your understanding of Spark Batch Processing concepts by working through a series of exercises that involve creating, running, and monitoring batch jobs in Azure Databricks.
Browse courses on
Azure Databricks
Show steps
Show steps
Show steps
- Create an Azure Databricks workspace
- Create a Spark DataFrame from a data source
- Perform data transformations using Spark SQL
- Write the transformed data to a data sink
- Monitor the job execution using the Azure Databricks UI
Build an ETL Pipeline for Batch Data Processing
Show steps
Reinforce your understanding of Azure Data Factory's capabilities by creating an ETL pipeline that transforms and loads data from one system to another.
Browse courses on
Azure Data Factory
Show steps
Show steps
Show steps
- Design the ETL pipeline architecture
- Create a data source and sink
- Transform the data using Data Factory transformations
- Schedule and execute the pipeline
- Monitor the pipeline execution
Hands-on Lab: Build a Batch Data Processing Solution with Azure Synapse Analytics
Show steps
Gain practical experience in building a batch data processing solution using Azure Synapse Analytics by following a hands-on lab that guides you through the entire process.
Browse courses on
Azure Synapse Analytics
Show steps
Show steps
Show steps
- Create an Azure Synapse workspace
- Ingest data from various sources
- Transform and process data using Synapse SQL
- Store the processed data in Azure Data Lake Storage
- Monitor and troubleshoot the data processing pipeline
Contribute to the Apache Spark Community
Show steps
Deepen your understanding of Spark Batch Processing and contribute to the broader community by actively participating in the Apache Spark project.
Browse courses on
Apache Spark
Show steps
Show steps
Show steps
- Identify areas within Apache Spark that align with your interests
- Join the Apache Spark community and mailing lists
- Review the Spark codebase and documentation
- Contribute bug fixes or enhancements to the project
- Engage in discussions and provide support to other community members
Develop a Batch Data Processing Solution for a Real-World Scenario
Show steps
Showcase your mastery of batch data processing by developing a comprehensive solution that addresses a specific business challenge or problem.
Show steps
Show steps
Show steps
- Define the business requirements and objectives
- Design the batch data processing architecture
- Select and configure appropriate Azure services
- Develop and implement the data processing pipeline
- Monitor, evaluate, and optimize the solution
Career center
Learners who complete Building Batch Data Processing Solutions in Microsoft Azure will develop knowledge and skills
that may be useful to these careers:
Data Architect
Data Architect
A Data Architect designs and builds data systems. This course will teach you how to use Azure products to perform batch data processing operations, which is a critical skill for Data Architects. By taking this course, you will gain the knowledge and skills needed to be successful in this role.
Cloud Architect
Cloud Architect
A Cloud Architect designs and builds cloud-based solutions. This course will teach you how to use Azure products to perform batch data processing operations, which is a critical skill for Cloud Architects. By taking this course, you will gain the knowledge and skills needed to be successful in this role.
Data Engineer
Data Engineer
A Data Engineer designs and implements data solutions in the cloud. This course will teach you how to use Azure products to perform batch data processing operations, which is a critical skill for Data Engineers. By taking this course, you will gain the knowledge and skills needed to be successful in this role.
Data Analyst
Data Analyst
A Data Analyst analyzes data to identify trends and patterns. This course will teach you how to use Azure products to perform batch data processing operations, which is a valuable skill for Data Analysts. By taking this course, you will gain the knowledge and skills needed to be successful in this role.
Database Administrator
Database Administrator
A Database Administrator manages and maintains databases. This course will teach you how to use Azure products to perform batch data processing operations, which is a valuable skill for Database Administrators. By taking this course, you will gain the knowledge and skills needed to be successful in this role.
Data Scientist
Data Scientist
A Data Scientist uses data to solve business problems. This course will teach you how to use Azure products to perform batch data processing operations, which is a valuable skill for Data Scientists. By taking this course, you will gain the knowledge and skills needed to be successful in this role.
Machine Learning Engineer
Machine Learning Engineer
A Machine Learning Engineer designs and builds machine learning models. This course will teach you how to use Azure products to perform batch data processing operations, which is a valuable skill for Machine Learning Engineers. By taking this course, you will gain the knowledge and skills needed to be successful in this role.
Software Engineer
Software Engineer
A Software Engineer designs, develops, and maintains software applications. This course will teach you how to use Azure products to perform batch data processing operations, which is a valuable skill for Software Engineers. By taking this course, you will gain the knowledge and skills needed to be successful in this role.
Business Intelligence Analyst
Business Intelligence Analyst
A Business Intelligence Analyst uses data to make business decisions. This course will teach you how to use Azure products to perform batch data processing operations, which is a valuable skill for Business Intelligence Analysts. By taking this course, you will gain the knowledge and skills needed to be successful in this role.
Data Governance Analyst
Data Governance Analyst
A Data Governance Analyst manages and controls data. This course may be helpful to Data Governance Analysts who want to learn how to use Azure products to perform batch data processing operations.
DevOps Engineer
DevOps Engineer
A DevOps Engineer automates the software development and deployment process. This course may be helpful to DevOps Engineers who want to learn how to use Azure products to perform batch data processing operations.
Security Analyst
Security Analyst
A Security Analyst protects computer systems from security breaches. This course may be helpful to Security Analysts who want to learn how to use Azure products to perform batch data processing operations.
System Administrator
System Administrator
A System Administrator manages and maintains computer systems. This course may be helpful to System Administrators who want to learn how to use Azure products to perform batch data processing operations.
Cloud Security Engineer
Cloud Security Engineer
A Cloud Security Engineer protects cloud-based systems from security breaches. This course may be helpful to Cloud Security Engineers who want to learn how to use Azure products to perform batch data processing operations.
Network Administrator
Network Administrator
A Network Administrator manages and maintains computer networks. This course may be helpful to Network Administrators who want to learn how to use Azure products to perform batch data processing operations.
For more career information including salaries, visit:
OpenCourser.com/course/792n3w/building
Reading list
We've selected five books
that we think will supplement your
learning. Use these to
develop background knowledge, enrich your coursework, and gain a
deeper understanding of the topics covered in
Building Batch Data Processing Solutions in Microsoft Azure.
Comprehensive guide to building data-intensive applications. It covers all aspects of data-intensive applications, from data modeling to performance tuning.
Is the definitive guide to Apache Hadoop, the open-source framework for big data processing. It covers all aspects of Hadoop, from installation to administration.
Is the definitive guide to Apache Spark, the open-source framework for big data processing. It covers all aspects of Spark, from installation to administration.
Save
Is the definitive guide to Apache ZooKeeper, the open-source distributed coordination service. It covers all aspects of ZooKeeper, from installation to administration.
Save
While this book is not specific to Microsoft Azure, it provides foundational concepts and principles of big data systems that are applicable to Azure platform.
For more information about how these books relate to this course, visit:
OpenCourser.com/course/792n3w/building
Share
Similar courses
Here are nine courses similar to
Building Batch Data Processing Solutions in Microsoft Azure.
DP-203: Data Ingestion and Preparation
OpenCourser.com/course/t3tat2/dp
DP-203: Data Ingestion and Preparation
Most relevant
DP-203: Processing in Azure Using Streaming Solutions
OpenCourser.com/course/53e3se/dp
DP-203: Processing in Azure Using Streaming Solutions
Most relevant
DP-203: Processing in Azure Using Batch Solutions
OpenCourser.com/course/rm8nnq/dp
DP-203: Processing in Azure Using Batch Solutions
Most relevant
Introduction to Azure Stream Analytics
OpenCourser.com/course/ef9hf0/introduction
Introduction to Azure Stream Analytics
Most relevant
DP-203: Building an Azure Data Engineer Foundation
OpenCourser.com/course/nwdu1j/dp
DP-203: Building an Azure Data Engineer Foundation
Most relevant
Implement Real Time Analytics using Azure Stream Analytics
OpenCourser.com/course/lrtmoy/implement
Implement Real Time Analytics using Azure Stream Analytics
Most relevant
Handling Batch Data with Apache Spark on Databricks
OpenCourser.com/course/1h7104/handling
Handling Batch Data with Apache Spark on Databricks
Most relevant
Create and Monitor Data Pipelines for a Batch Processing Solution
OpenCourser.com/course/r7f1q4/create
Create and Monitor Data Pipelines for a Batch Processing...
Most relevant
Conceptualizing the Processing Model for Apache Flink
OpenCourser.com/course/ykkn6w/conceptualizing
Conceptualizing the Processing Model for Apache Flink
Most relevant
Time
105 hours
Level
Intermediate
Via
Pluralsight
Instructor
Tim Warner
Language
English
Good to know
Teaches skills, knowledge, and tools that are highly relevant to industry
Teaches skills, knowledge, and tools that are core for Azure Data Engineers
Develops skills for performing high-scale ETL and ELT operations in the Azure public cloud
Covers event processing with Azure Stream Analytics
Examines hosted Apache Spark processing with Azure Databricks
Explores data transfer in bulk with Azure Data Factory and Azure Data Explorer
Our mission
OpenCourser helps millions of learners each year. People visit us to learn workspace skills, ace their exams, and nurture their curiosity.
Our extensive catalog contains over 50,000 courses and twice as many books. Browse by search, by topic, or even by career interests. We'll match you to the right resources quickly.
Find this site helpful? Tell a friend about us.
Affiliate disclosure
We're supported by our community of learners. When you purchase or subscribe to courses and programs or purchase books, we may earn a commission from our partners.
Your purchases help us maintain our catalog and keep our servers humming without ads.
Thank you for supporting OpenCourser.
© 2016 - 2024 OpenCourser