OpenCourser brand icon
Sign in
We may earn an affiliate commission when you visit our partners.
Course image
Udemy logo

The Complete Apache Kafka Practical Guide

Bogdan Stashchuk | Software Engineer, MBA, PhD

This course is all about Apache Kafka

Understand HOW Apache Kafka works and learn its core features ON PRACTICE. This is 80% practice course without no useless demos.

Read more

This course is all about Apache Kafka

Understand HOW Apache Kafka works and learn its core features ON PRACTICE. This is 80% practice course without no useless demos.

Build custom Apache Kafka Producers and Consumers using native Java API. Also you will build projects using APIs for other programming languages like Node.js and Python.

Become a master of Apache Kafka by understanding and practicing its architecture and main features.

All project files and mostly used commands are available in the GitHub repository.

This is the most complete practical Apache Kafka guide here on Udemy that includes tons of practical activities. Most important is that you will learn how Apache Kafka works and knowing it you will be able much more easier use its features and fix mistakes in the future after you'll finish this course. You can have zero knowledge about Apache Kafka, Java, Node.js or Python. All will be taught from scratch, from basic to advanced features. If you want to get deep knowledge of Apache Kafka this course is for you.

We will start by installing Apache Kafka on your computer, VPS (Virtual Private Server) or Virtual Machine on your computer. You will learn that installation of Apache Kafka is pretty easy - you just need to download archive with executable scripts and run them. Also you will learn and practice how to run multiple brokers on the same computer.

Afterwards we will jump into tons of practice activities and use different Apache Kafka features and built-in scripts. You will launch Zookeeper, multiple Brokers, Console Consumer and Console Producer. Also you will test performance of the Kafka Cluster using built-in utility called Performance Monitor.

In practice sections you will perform multiple practice Apache activities:

  1. Create cluster with multiple brokers

  2. Create topic with multiple partitions spread across different brokers

  3. Create topics with replication factor that allows you to store copy of every message on different brokers for redundancy

  4. Produce messages using built-in Console Producer

  5. Consume messages using built-in Console Consumer

  6. Launch multiple consumers in the same consumer group

  7. Launch Performance Monitor for testing Consumers and Producers performance and speed

You will also learn and practice how to use Apache Kafka API to create your own Consumers and Producers

  1. Create Java Maven project

  2. Launch Producer and Consumer using Java

  3. Launch multiple consumers in the same Consumer Group

  4. Understand and practice difference between "subscribe" and "assign"

  5. Create Node.js project

  6. Launch Producers and Consumers using Node.js

  7. Create Python project

  8. Launch Producers and Consumers using Python

During the course you will need to view and edit text files. For that you will NOT use terminal. Instead you will use GUI application VisualStudio Code. Also all configuration files and project files are available in the GitHub repository. This means that during this course you will also learn how to use :

  • Git and GitHub

  • VisualStudio Code

With this course you will get lifetime-long access to more than 100 lectures and tens of practical exercises. After the course you will become a guru of Apache Kafka and will be able easily create custom Apache Kafka Producers and Consumers.

But most important is that you will

You will also get 30-days money-back guarantee. No questions asked.

Don't wait and join the course now.

Enroll now

14 deals to help you save

We found 14 deals and offers that may be relevant to this course.
Save money when you learn. All coupon codes, vouchers, and discounts are applied automatically unless otherwise noted.
Use code at checkout. Valid until December 3
Cyber Monday
Develop career-changing skills with these steeply discounted courses.
Up to
85%
off
BFCMSALE24
Use code at checkout. Only 3 days left!
Black Friday
Develop career-changing skills with these steeply discounted courses.
Up to
85%
off
BFCMSALE24
Use code at checkout. Valid until December 1
For new customers
Save when you purchase top courses. For new customers only.
Special Offer
UDEAFNULP2024
Valid for a limited time only
Future-proof your career
Access O'Reilly books, live events, courses, and more. Save with an annual subscription.
Take
15%
off

What's inside

Learning objectives

  • Understand how apache kafka works
  • Create fault-tolerant clusters with topics replication across multiple brokers
  • Apache kafka architecture
  • How to create kafka cluster with multiple brokers
  • Practice to create producers and consumers using built-in console producer and console consumer
  • Write custom producer and consumer in java
  • Use python and node to produce and consume messages
  • Learn components of the apache kafka cluster

Syllabus

Introduction
Let's get connected! Join the Learning Community
Apache Kafka Installation Overview

This lecture I'll briefly overview set-up of Apache Kafka on different computers

Read more
Installing Apache Kafka on the Mac and Unix-like systems
Installing Apache Kafka on the Mac
Installing Ubuntu on MacOS using VirtualBox
Installing Apache Kafka on the remote Ubuntu server
SECTION 2 Introduction
Creating remote Ubuntu Virtual Private Server
Installing Apache Kafka on Virtual Private Server
Installing Apache Kafka on Windows
SECTION 3 Introduction
Starting Zookeeper and Kafka server on Windows
Installing Ubuntu on Windows using VirtualBox
Installing Apache Kafka on Ubuntu using GUI
Starting Apache Zookeeper and Kafka Broker
SECTION 4 Introduction
Observing contents of the Kafka folder
Reading and editing Kafka files using VisualStudio Code
Trying to start Kafka Server
Observing Kafka Server logs
Starting Zookeeper
Starting Kafka Server while Zookeeper is up and running
Observing logs folder and current kafka server setup
Creating and exploring Kafka Topic
SECTION 5 Introduction
How to connect to Kafka cluster
Create new Kafka topic
What happened after creation of the new topic
Read details about topic
Producing and consuming Messages
SECTION 6 Introduction
Send some messages using Kafka Console Producer
Consuming messages using Kafka Console Consumer
Consuming messages from the beginning
Running multiple consumers
Running multiple producers
What was changed in the Kafka logs
What is Apache Kafka and how it works
SECTION 7 Introduction
What is Apache Kafka
Broker
Broker cluster
Zookeeper
Zookeeper ensemble
Multiple Kafka clusters
Default ports of Zookeeper and Broker
Kafka Topic
Message structure
Topics and Partitions
Spreading messages across partitions
Partition Leader and Followers
Controller and it's responsibilities
How Producers write messages to the topic
How Consumers read messages from the topic
GitHub Repository and Diagrams for the course
SECTION 8 Introduction
GitHub repository and list of basic Kafka commands
Diagrams for the course
EXAMPLE 1 - Topic with Multiple Partitions
SECTION 9 Introduction
If you use remote Kafka brokers
Cleaning up existing kafka installation
Creating topic with multiple partitions
How messages were spread across different partitions
Reading messages from specific partition
Reading messages from specific offset in specific partition
Reading details about topic and __consumer_offsets topic
Summary for multiple partitions example
EXAMPLE 2 - Kafka Cluster with Multiple Brokers
SECTION 10 Introduction
Example overview - run multiple brokers
Creating separate configuration files for brokers
Launching three brokers
Getting cluster information and broker details from Zookeeper
Creating multiple-partition topic in the Kafka cluster
Looking at logs folders of every broker
Producing and consuming messages in the cluster
Details about topic in the cluster
Simulating broker failure in the cluster
Summary for broker cluster and topic without replication
EXAMPLE 3 - Multiple Brokers and Topic with Replication
SECTION 11 Introduction
Preparing for the next example with replication
Launching brokers and creating topic with replication
Observing logs folder and details of the topic
Producing and consuming messages in the topic with replication
Observing how messages were stored in the partitions on different brokers
Bringing down one of three brokers and observing changes
Bringing down another broker in the cluster
Bringing back both brokers
Summary for replication
EXAMPLE 4 - Kafka Consumer Groups
SECTION 12 Introduction
Example with consumer groups overview
Exploring default consumer groups
Starting consumer in the custom consumer group
Starting second consumer in the same consumer group

Good to know

Know what's good
, what to watch for
, and possible dealbreakers
Helps learners develop a deep mastery of Apache Kafka
Essentially practice-based, it offers only the most salient information
Appropriate for beginners, with clear explanations of the subject matter
Teaches core concepts, such as topics, partitions, replication, and consumer groups
Provides real-world examples and use cases
Offers guidance on troubleshooting and debugging common errors

Save this course

Save The Complete Apache Kafka Practical Guide to your list so you can find it easily later:
Save

Activities

Be better prepared before your course. Deepen your understanding during and after it. Supplement your coursework and achieve mastery of the topics covered in The Complete Apache Kafka Practical Guide with these activities:
Review Java programming basics
Reviewing Java programming basics will help you refresh your knowledge of the language and prepare you for the coding exercises in this course.
Browse courses on Java
Show steps
  • Go through online tutorials or documentation
  • Practice writing simple Java programs
Review basic data structures and algorithms
Reviewing basic data structures and algorithms will refresh your knowledge of these essential concepts and prepare you for the more advanced topics covered in this course.
Browse courses on Data Structures
Show steps
  • Go through online tutorials or documentation
  • Solve practice problems
Organize your course notes and materials
Keeping your course notes and materials organized will help you stay on track and easily find the information you need.
Show steps
  • Create a dedicated folder for your course materials
  • File your notes, assignments, and other materials
Six other activities
Expand to see all activities and additional details
Show all nine activities
Install and configure Apache Kafka on your local machine
Getting Apache Kafka up and running on your local machine will allow you to practice the core features of Kafka and experiment with different configurations.
Browse courses on Java
Show steps
  • Download the Apache Kafka distribution
  • Extract the distribution and set the environment variables
  • Start the Zookeeper server
  • Start the Kafka broker
  • Create a topic and produce and consume messages
Join a Kafka study group or community
Joining a Kafka study group or community will provide you with opportunities to connect with other Kafka users, ask questions, and share knowledge.
Browse courses on Kafka
Show steps
  • Search for Kafka study groups or communities online
  • Join a group or community that aligns with your interests
  • Participate in discussions and ask questions
Build a Kafka producer and consumer using Java
Building a Kafka producer and consumer from scratch will help you understand the inner workings of Kafka and how to use its APIs.
Browse courses on Java
Show steps
  • Create a new Java project
  • Add the necessary Kafka dependencies
  • Create a Kafka producer
  • Create a Kafka consumer
  • Produce and consume messages
Write a blog post or article about Apache Kafka
Writing a blog post or article about Apache Kafka will help you solidify your understanding of the technology and share your knowledge with others.
Browse courses on Apache Kafka
Show steps
  • Choose a topic related to Apache Kafka
  • Research the topic and gather information
  • Write the blog post or article
  • Publish your content
Contribute to the Apache Kafka project
Contributing to the Apache Kafka project will give you hands-on experience with Kafka's codebase, development process, and community.
Browse courses on Apache Kafka
Show steps
  • Find a project or issue to work on
  • Set up your development environment
  • Implement your changes
  • Submit a pull request
  • Review feedback and make revisions
Develop a data pipeline using Kafka
Building a data pipeline using Kafka will allow you to apply your Kafka skills to a real-world scenario and demonstrate your understanding of data engineering concepts.
Browse courses on Data Pipeline
Show steps
  • Design the data pipeline architecture
  • Implement the Kafka producer and consumer
  • Configure the Kafka cluster
  • Monitor the data pipeline
  • Document the data pipeline

Career center

Learners who complete The Complete Apache Kafka Practical Guide will develop knowledge and skills that may be useful to these careers:

Reading list

We haven't picked any books for this reading list yet.

Share

Help others find this course page by sharing it with your friends and followers:
Facebook
LinkedIn
Reddit
X
Link
Email

Similar courses

Here are nine courses similar to The Complete Apache Kafka Practical Guide.
Apache Kafka Deep Dive
Most relevant
Apache Kafka Series - Learn Apache Kafka for Beginners v3
Most relevant
Apache Kafka Series - Confluent Schema Registry & REST...
Most relevant
ETL and Data Pipelines with Shell, Airflow and Kafka
Most relevant
Apache Kafka for Event-Driven Spring Boot Microservices
Most relevant
Using Kafka on Confluent
Most relevant
Protective Technology with Apache Kafka
Most relevant
Enforcing Data Contracts with Kafka Schema Registry
Most relevant
Kafka Connect Fundamentals
Most relevant
Course image
Effort
8.5 total hours
Via
Udemy
Instructor
Bogdan Stashchuk | Software Engineer, MBA, PhD
Language
English
Good to know
Helps learners develop a deep mastery of Apache Kafka
Essentially practice-based, it offers only the most salient information
Appropriate for beginners, with clear explanations of the subject matter
Teaches core concepts, such as topics, partitions, replication, and consumer groups
Provides real-world examples and use cases
Offers guidance on troubleshooting and debugging common errors
Share and help others discover this course.
Facebook LinkedIn X Reddit Link Email
Don't miss out
Enroll today to gain access to this course
Enroll in this course
Our mission

OpenCourser helps millions of learners each year. People visit us to learn workspace skills, ace their exams, and nurture their curiosity.

Our extensive catalog contains over 50,000 courses and twice as many books. Browse by search, by topic, or even by career interests. We'll match you to the right resources quickly.

Find this site helpful? Tell a friend about us.

Affiliate disclosure

We're supported by our community of learners. When you purchase or subscribe to courses and programs or purchase books, we may earn a commission from our partners.

Your purchases help us maintain our catalog and keep our servers humming without ads.

Thank you for supporting OpenCourser.

© 2016 - 2024 OpenCourser