We may earn an affiliate commission when you visit our partners.

The Complete Hands-On Introduction to Apache Airflow

Marc Lamberti

4.5

Based on 11,595 ratings

, see reviews

Marc Lamberti

Apache Airflow is an open-source platform to programmatically author, schedule and monitor workflows. If you have many ETL(s) to manage, Airflow is a must-have.

In this course you are going to learn everything you need to start using Apache Airflow through theory and pratical videos. Starting from very basic notions such as, what is Airflow and how it works, we will dive into advanced concepts such as, how to create plugins and make real dynamic pipelines.

Enroll now

Or start a personal plan

And upskill with Udemy

Here's a deal for you

We found an offer that may be relevant to this course.

Save money when you learn. All coupon codes, vouchers, and discounts are applied automatically unless otherwise noted.

Only 4 hours left!

Get ahead, stay ahead

Save on courses you need to learn the skills you need to get ahead

Take

85%

off

What's inside

Learning objectives

Create plugins to add functionalities to apache airflow.
Using docker with airflow and different executors
Master core functionalities such as dags, operators, tasks, workflows, etc
Understand and apply advanced concepts of apache airflow such as xcoms, branching and subdags.

The difference between sequential, local and celery executors, how do they work and how can you use them.
Use apache airflow in a big data ecosystem with hive, postgresql, elasticsearch etc.
Install and configure apache airflow
Think, answer and implement solutions using airflow to real data processing problems

Create plugins to add functionalities to apache airflow.
Using docker with airflow and different executors
Master core functionalities such as dags, operators, tasks, workflows, etc
Understand and apply advanced concepts of apache airflow such as xcoms, branching and subdags.
The difference between sequential, local and celery executors, how do they work and how can you use them.
Use apache airflow in a big data ecosystem with hive, postgresql, elasticsearch etc.
Install and configure apache airflow
Think, answer and implement solutions using airflow to real data processing problems

Syllabus

Just a quick introduction.

Prerequisites

Course Objectives

Who I am?

Traffic lights

Read about what's good

what should give you pause

and possible dealbreakers

Expands your knowledge of Apache Airflow, a vital tool in ETL management

Taught by Marc Lamberti, a recognized expert in Apache Airflow

Builds a foundation for using plugins to expand Airflow functionality

Demonstrates effective use of Docker with Airflow and various executors

Develops skills in creating, customizing, and managing Apache Airflow DAGs

Reinforces knowledge of Apache Airflow's core functionalities

Save this course

Create your own learning path. Save this course to your list so you can find it easily later.

Save

Reviews summary

Hands-on airflow course

According to students, this in-depth course provides a state-of-art look at Apache Airflow. Beyond the beginner level material, this course teaches real-world applications of Airflow and expert practices. Students say that the instructor is experienced with the material.

Learners praise the advanced material including real-world applications.

"The instructor seems to have a lot of experience with this tool, you can check out his blog + I recommend buying a new course from Marc: "the-ultimate-hands-on-course-to-master-apache-airflow". It goes much more in depth and shows state-of-the-art options to deploy Airflow with KubernetesExecutor."

The course instructor is praised by students for their deep knowledge of the material.

Students say that this course covers advanced material.

"A very good introduction to Airflow."

Activities

Be better prepared before your course. Deepen your understanding during and after it. Supplement your coursework and achieve mastery of the topics covered in The Complete Hands-On Introduction to Apache Airflow with these activities:

Review the book 'Designing Data-Intensive Applications'

Show steps

Review key concepts in data engineering and big data to strengthen your understanding of the topics covered in this course.

View Secret Colors on Amazon

Show steps

Read the book's introduction and first three chapters.
Summarize the main points of each chapter in your own words.
Identify any areas where you need further clarification.
Discuss the book's concepts with a study partner or online forum.

Solve Apache Airflow coding challenges

Show steps

Sharpen your Apache Airflow skills by solving coding challenges that test your understanding of core concepts and syntax.

Browse courses on Apache Airflow

Show steps

Find coding challenges on platforms like LeetCode or HackerRank.
Attempt to solve the challenges on your own.
Review the solutions and identify areas for improvement.

Follow online tutorials on Apache Airflow

Show steps

Supplement your course learning with online tutorials that provide step-by-step guidance on specific Apache Airflow topics.

Show steps

Identify areas where you need additional support.
Find reputable online tutorials that cover those topics.
Follow the tutorials carefully and implement the examples.

Four other activities

Expand to see all activities and additional details

Show all seven activities

Create a data pipeline using Apache Airflow

Show steps

Build a practical understanding of Apache Airflow by implementing a data pipeline for a real-world scenario.

Show steps

Choose a data source and define the data processing tasks.
Design the Airflow DAG and define the operators.
Implement the DAG and test its functionality.
Deploy the pipeline and monitor its performance.

Attend Apache Airflow meetups or conferences

Show steps

Connect with other Apache Airflow users and experts to exchange knowledge and learn about best practices.

Show steps

Find Apache Airflow meetups or conferences in your area.
Register for the event and attend the sessions.
Network with other attendees and share your experiences.

Build a personal Apache Airflow project

Show steps

Apply your Apache Airflow knowledge by building a project that solves a real-world problem or explores a specific topic.

Show steps

Define the scope and goals of your project.
Design the architecture and implementation plan.
Develop and test your project.
Deploy and maintain your project.

Contribute to the Apache Airflow community

Show steps

Gain hands-on experience and give back to the Apache Airflow community by contributing to projects or forums.

Show steps

Join the Apache Airflow community forums.
Identify areas where you can contribute, such as answering questions or reporting issues.
Submit your contributions and engage with other community members.

Career center

Learners who complete The Complete Hands-On Introduction to Apache Airflow will develop knowledge and skills that may be useful to these careers:

Data Engineer

A Data Engineer designs, builds, and maintains data pipelines. They work within big data ecosystems to build out data infrastructure. Often, Data Engineers will also develop solutions using Apache Airflow. This course will allow you to build the foundation you need to become a Data Engineer by helping you build out the skills needed to use Airflow to create plugins, use Docker with Airflow, use core functionalities, and use advanced concepts. This course may help you get into or advance your Data Engineering career.

See salaries and explore the career path for Data Engineer

Data Analyst

A Data Analyst transforms raw data into insights that companies can use to make better decisions. They build out dashboards and data visualizations, which can be built using Apache Airflow. This course will allow you to build the foundation you need to become a Data Analyst by helping you build out the skills needed to use Airflow to create plugins, use Docker with Airflow, use core functionalities, and use advanced concepts. This course may help you get into or advance your Data Analyst career.

See salaries and explore the career path for Data Analyst

Software Engineer

A Software Engineer designs, develops, and maintains software systems. They work within a variety of industries, and often use Apache Airflow. This course will allow you to build the foundation you need to become a Software Engineer by helping you build out the skills needed to use Airflow to create plugins, use Docker with Airflow, use core functionalities, and use advanced concepts. This course may help you get into or advance your Software Engineering career.

See salaries and explore the career path for Software Engineer

Data Scientist

A Data Scientist uses scientific methods to extract knowledge and insights from data. They use Apache Airflow to build out data pipelines and manage data. This course will allow you to build the foundation you need to become a Data Scientist by helping you build out the skills needed to use Airflow to create plugins, use Docker with Airflow, use core functionalities, and use advanced concepts. This course may help you get into or advance your Data Science career.

See salaries and explore the career path for Data Scientist

Machine Learning Engineer

A Machine Learning Engineer designs, builds, and maintains machine learning systems. They work within various industries, and often use Apache Airflow. This course will allow you to build the foundation you need to become a Machine Learning Engineer by helping you build out the skills needed to use Airflow to create plugins, use Docker with Airflow, use core functionalities, and use advanced concepts. This course may help you get into or advance your Machine Learning Engineering career.

See salaries and explore the career path for Machine Learning Engineer

Business Analyst

A Business Analyst analyzes business processes and develops solutions to improve efficiency. They work within various industries, and often use Apache Airflow. This course will allow you to build the foundation you need to become a Business Analyst by helping you build out the skills needed to use Airflow to create plugins, use Docker with Airflow, use core functionalities, and use advanced concepts. This course may help you get into or advance your Business Analyst career.

See salaries and explore the career path for Business Analyst

Cloud Engineer

A Cloud Engineer designs, builds, and maintains cloud computing systems. They work within various industries, and often use Apache Airflow. This course will allow you to build the foundation you need to become a Cloud Engineer by helping you build out the skills needed to use Airflow to create plugins, use Docker with Airflow, use core functionalities, and use advanced concepts. This course may help you get into or advance your Cloud Engineering career.

See salaries and explore the career path for Cloud Engineer

Database Administrator

A Database Administrator designs, builds, and maintains database systems. They work within various industries, and often use Apache Airflow. This course will allow you to build the foundation you need to become a Database Administrator by helping you build out the skills needed to use Airflow to create plugins, use Docker with Airflow, use core functionalities, and use advanced concepts. This course may help you get into or advance your Database Administrator career.

See salaries and explore the career path for Database Administrator

Data Architect

A Data Architect designs, builds, and maintains data systems. They work within various industries, and often use Apache Airflow. This course will allow you to build the foundation you need to become a Data Architect by helping you build out the skills needed to use Airflow to create plugins, use Docker with Airflow, use core functionalities, and use advanced concepts. This course may help you get into or advance your Data Architect career.

See salaries and explore the career path for Data Architect

Project Manager

A Project Manager plans, executes, and closes projects. They work within various industries, and often use Apache Airflow. This course will allow you to build the foundation you need to become a Project Manager by helping you build out the skills needed to use Airflow to create plugins, use Docker with Airflow, use core functionalities, and use advanced concepts. This course may help you get into or advance your Project Management career.

See salaries and explore the career path for Project Manager

Product Manager

A Product Manager plans, develops, and launches products. They work within various industries, and often use Apache Airflow. This course will allow you to build the foundation you need to become a Product Manager by helping you build out the skills needed to use Airflow to create plugins, use Docker with Airflow, use core functionalities, and use advanced concepts. This course may help you get into or advance your Product Management career.

See salaries and explore the career path for Product Manager

Data Integration Specialist

A Data Integration Specialist integrates data from different sources into a single system. They work within various industries, and often use Apache Airflow. This course will allow you to build the foundation you need to become a Data Integration Specialist by helping you build out the skills needed to use Airflow to create plugins, use Docker with Airflow, use core functionalities, and use advanced concepts. This course may help you get into or advance your Data Integration Specialist career.

See salaries and explore the career path for Data Integration Specialist

ETL Developer

An ETL Developer designs, builds, and maintains ETL (Extract, Transform, Load) systems. They work within various industries, and often use Apache Airflow. This course will allow you to build the foundation you need to become an ETL Developer by helping you build out the skills needed to use Airflow to create plugins, use Docker with Airflow, use core functionalities, and use advanced concepts. This course may help you get into or advance your ETL Developer career.

See salaries and explore the career path for ETL Developer

Big Data Engineer

A Big Data Engineer designs, builds, and maintains big data systems. They work within various industries, and often use Apache Airflow. This course will allow you to build the foundation you need to become a Big Data Engineer by helping you build out the skills needed to use Airflow to create plugins, use Docker with Airflow, use core functionalities, and use advanced concepts. This course may help you get into or advance your Big Data Engineering career.

See salaries and explore the career path for Big Data Engineer

Data Analytics Manager

A Data Analytics Manager plans, executes, and manages data analytics projects. They work within various industries, and often use Apache Airflow. This course will allow you to build the foundation you need to become a Data Analytics Manager by helping you build out the skills needed to use Airflow to create plugins, use Docker with Airflow, use core functionalities, and use advanced concepts. This course may help you get into or advance your Data Analytics Management career.

See salaries and explore the career path for Data Analytics Manager

Reading list

We've selected five books that we think will supplement your learning. Use these to develop background knowledge, enrich your coursework, and gain a deeper understanding of the topics covered in The Complete Hands-On Introduction to Apache Airflow.

Designing Data-Intensive Applications

Save

Provides a broader perspective on data-intensive applications, including architectural patterns and best practices that can be applied to Apache Airflow.

Designing Data-Intensive Applications: The Big...

Paperback

Designing Data-Intensive Applications: The Big...

Kindle Edition

Python for Data Analysis

Save

Provides an introduction to Python and its libraries for data analysis, which are essential for working with Apache Airflow.

Python for Data Analysis: Data Wrangling with...

Paperback

Python for Data Analysis

Kindle Edition

Python for Data Analysis: Data Wrangling with...

Paperback

$$$

Abeloff's Clinical Oncology E-Book

Save

Comprehensive guide to using Airflow to build and manage data pipelines. It covers topics such as data ingestion, data processing, and data analysis. It valuable resource for anyone who wants to use Airflow to build and manage data pipelines.

Abeloff's Clinical Oncology Review E-Book

Kindle Edition

$$$

Middleton's Allergy 2-Volume Set

Save

Middleton's Allergy 2-Volume Set: Principles and...

Hardcover

$$$$

Haynes Buick Oldsmobile Pontiac Fullsize FWD, 1985...

Save

Guide to using Apache Airflow for deep learning. It covers topics such as using Airflow to build and manage deep learning pipelines, using Airflow with different deep learning tools and frameworks, and deploying Airflow in production.

Haynes Buick Oldsmobile Pontiac Fullsize FWD, 1985...

Paperback

Check price

Haynes Buick Oldsmobile Pontiac Fullsize FWD, 1985...

Kindle Edition

Check price

Help others find this course page by sharing it with your friends and followers: