The Complete Apache Kafka Practical Guide from Udemy

This course is all about Apache Kafka

Understand HOW Apache Kafka works and learn its core features ON PRACTICE. This is 80% practice course without no useless demos.

Build custom Apache Kafka Producers and Consumers using native Java API. Also you will build projects using APIs for other programming languages like Node.js and Python.

Become a master of Apache Kafka by understanding and practicing its architecture and main features.

All project files and mostly used commands are available in the GitHub repository.

This is the most complete practical Apache Kafka guide here on Udemy that includes tons of practical activities. Most important is that you will learn how Apache Kafka works and knowing it you will be able much more easier use its features and fix mistakes in the future after you'll finish this course. You can have zero knowledge about Apache Kafka, Java, Node.js or Python. All will be taught from scratch, from basic to advanced features. If you want to get deep knowledge of Apache Kafka this course is for you.

We will start by installing Apache Kafka on your computer, VPS (Virtual Private Server) or Virtual Machine on your computer. You will learn that installation of Apache Kafka is pretty easy - you just need to download archive with executable scripts and run them. Also you will learn and practice how to run multiple brokers on the same computer.

Afterwards we will jump into tons of practice activities and use different Apache Kafka features and built-in scripts. You will launch Zookeeper, multiple Brokers, Console Consumer and Console Producer. Also you will test performance of the Kafka Cluster using built-in utility called Performance Monitor.

In practice sections you will perform multiple practice Apache activities:

Create cluster with multiple brokers
Create topic with multiple partitions spread across different brokers
Create topics with replication factor that allows you to store copy of every message on different brokers for redundancy
Produce messages using built-in Console Producer
Consume messages using built-in Console Consumer
Launch multiple consumers in the same consumer group
Launch Performance Monitor for testing Consumers and Producers performance and speed

You will also learn and practice how to use Apache Kafka API to create your own Consumers and Producers

Create Java Maven project
Launch Producer and Consumer using Java
Launch multiple consumers in the same Consumer Group
Understand and practice difference between "subscribe" and "assign"
Create Node.js project
Launch Producers and Consumers using Node.js
Create Python project
Launch Producers and Consumers using Python

During the course you will need to view and edit text files. For that you will NOT use terminal. Instead you will use GUI application VisualStudio Code. Also all configuration files and project files are available in the GitHub repository. This means that during this course you will also learn how to use :

Git and GitHub
VisualStudio Code

With this course you will get lifetime-long access to more than 100 lectures and tens of practical exercises. After the course you will become a guru of Apache Kafka and will be able easily create custom Apache Kafka Producers and Consumers.

But most important is that you will

You will also get 30-days money-back guarantee. No questions asked.

Don't wait and join the course now.

What's inside

Learning objectives

Understand how apache kafka works
Create fault-tolerant clusters with topics replication across multiple brokers
Apache kafka architecture
How to create kafka cluster with multiple brokers

Practice to create producers and consumers using built-in console producer and console consumer
Write custom producer and consumer in java
Use python and node to produce and consume messages
Learn components of the apache kafka cluster

Understand how apache kafka works
Create fault-tolerant clusters with topics replication across multiple brokers
Apache kafka architecture
How to create kafka cluster with multiple brokers
Practice to create producers and consumers using built-in console producer and console consumer
Write custom producer and consumer in java
Use python and node to produce and consume messages
Learn components of the apache kafka cluster

Syllabus

Introduction

Let's get connected! Join the Learning Community

Apache Kafka Installation Overview

This lecture I'll briefly overview set-up of Apache Kafka on different computers

Installing Apache Kafka on the Mac and Unix-like systems

Installing Apache Kafka on the Mac

Installing Ubuntu on MacOS using VirtualBox

Installing Apache Kafka on the remote Ubuntu server

SECTION 2 Introduction

Creating remote Ubuntu Virtual Private Server

Installing Apache Kafka on Virtual Private Server

Installing Apache Kafka on Windows

SECTION 3 Introduction

Starting Zookeeper and Kafka server on Windows

Installing Ubuntu on Windows using VirtualBox

Installing Apache Kafka on Ubuntu using GUI

Starting Apache Zookeeper and Kafka Broker

SECTION 4 Introduction

Observing contents of the Kafka folder

Reading and editing Kafka files using VisualStudio Code

Trying to start Kafka Server

Observing Kafka Server logs

Starting Zookeeper

Starting Kafka Server while Zookeeper is up and running

Observing logs folder and current kafka server setup

Creating and exploring Kafka Topic

SECTION 5 Introduction

How to connect to Kafka cluster

Create new Kafka topic

What happened after creation of the new topic

Read details about topic

Producing and consuming Messages

SECTION 6 Introduction

Send some messages using Kafka Console Producer

Consuming messages using Kafka Console Consumer

Consuming messages from the beginning

Running multiple consumers

Running multiple producers

What was changed in the Kafka logs

What is Apache Kafka and how it works

SECTION 7 Introduction

What is Apache Kafka

Broker

Broker cluster

Zookeeper

Zookeeper ensemble

Multiple Kafka clusters

Default ports of Zookeeper and Broker

Kafka Topic

Message structure

Topics and Partitions

Spreading messages across partitions

Partition Leader and Followers

Controller and it's responsibilities

How Producers write messages to the topic

How Consumers read messages from the topic

GitHub Repository and Diagrams for the course

SECTION 8 Introduction

GitHub repository and list of basic Kafka commands

Diagrams for the course

EXAMPLE 1 - Topic with Multiple Partitions

SECTION 9 Introduction

If you use remote Kafka brokers

Cleaning up existing kafka installation

Creating topic with multiple partitions

How messages were spread across different partitions

Reading messages from specific partition

Reading messages from specific offset in specific partition

Reading details about topic and __consumer_offsets topic

Summary for multiple partitions example

EXAMPLE 2 - Kafka Cluster with Multiple Brokers

SECTION 10 Introduction

Example overview - run multiple brokers

Creating separate configuration files for brokers

Launching three brokers

Getting cluster information and broker details from Zookeeper

Creating multiple-partition topic in the Kafka cluster

Looking at logs folders of every broker

Producing and consuming messages in the cluster

Details about topic in the cluster

Simulating broker failure in the cluster

Summary for broker cluster and topic without replication

EXAMPLE 3 - Multiple Brokers and Topic with Replication

SECTION 11 Introduction

Preparing for the next example with replication

Launching brokers and creating topic with replication

Observing logs folder and details of the topic

Producing and consuming messages in the topic with replication

Observing how messages were stored in the partitions on different brokers

Bringing down one of three brokers and observing changes

Bringing down another broker in the cluster

Bringing back both brokers

Summary for replication

EXAMPLE 4 - Kafka Consumer Groups

SECTION 12 Introduction

Example with consumer groups overview

Exploring default consumer groups

Starting consumer in the custom consumer group

Starting second consumer in the same consumer group

Good to know

Know what's good

, what to watch for

, and possible dealbreakers

Helps learners develop a deep mastery of Apache Kafka

Essentially practice-based, it offers only the most salient information

Appropriate for beginners, with clear explanations of the subject matter

Teaches core concepts, such as topics, partitions, replication, and consumer groups

Provides real-world examples and use cases

Offers guidance on troubleshooting and debugging common errors

Activities

Be better prepared before your course. Deepen your understanding during and after it. Supplement your coursework and achieve mastery of the topics covered in The Complete Apache Kafka Practical Guide with these activities:

Review Java programming basics

Show steps

Reviewing Java programming basics will help you refresh your knowledge of the language and prepare you for the coding exercises in this course.

Browse courses on Java

Show steps

Go through online tutorials or documentation
Practice writing simple Java programs

Review basic data structures and algorithms

Show steps

Reviewing basic data structures and algorithms will refresh your knowledge of these essential concepts and prepare you for the more advanced topics covered in this course.

Browse courses on Data Structures

Show steps

Go through online tutorials or documentation
Solve practice problems

Organize your course notes and materials

Show steps

Keeping your course notes and materials organized will help you stay on track and easily find the information you need.

Show steps

Create a dedicated folder for your course materials
File your notes, assignments, and other materials

Six other activities

Expand to see all activities and additional details

Show all nine activities

Install and configure Apache Kafka on your local machine

Show steps

Getting Apache Kafka up and running on your local machine will allow you to practice the core features of Kafka and experiment with different configurations.

Browse courses on Java

Show steps

Download the Apache Kafka distribution
Extract the distribution and set the environment variables
Start the Zookeeper server
Start the Kafka broker
Create a topic and produce and consume messages

Join a Kafka study group or community

Show steps

Joining a Kafka study group or community will provide you with opportunities to connect with other Kafka users, ask questions, and share knowledge.

Browse courses on Kafka

Show steps

Search for Kafka study groups or communities online
Join a group or community that aligns with your interests
Participate in discussions and ask questions

Build a Kafka producer and consumer using Java

Show steps

Building a Kafka producer and consumer from scratch will help you understand the inner workings of Kafka and how to use its APIs.

Browse courses on Java

Show steps

Create a new Java project
Add the necessary Kafka dependencies
Create a Kafka producer
Create a Kafka consumer
Produce and consume messages

Write a blog post or article about Apache Kafka

Show steps

Writing a blog post or article about Apache Kafka will help you solidify your understanding of the technology and share your knowledge with others.

Browse courses on Apache Kafka

Show steps

Choose a topic related to Apache Kafka
Research the topic and gather information
Write the blog post or article
Publish your content

Contribute to the Apache Kafka project

Show steps

Contributing to the Apache Kafka project will give you hands-on experience with Kafka's codebase, development process, and community.

Browse courses on Apache Kafka

Show steps

Find a project or issue to work on
Set up your development environment
Implement your changes
Submit a pull request
Review feedback and make revisions

Develop a data pipeline using Kafka

Show steps

Building a data pipeline using Kafka will allow you to apply your Kafka skills to a real-world scenario and demonstrate your understanding of data engineering concepts.

Browse courses on Data Pipeline

Show steps

Design the data pipeline architecture
Implement the Kafka producer and consumer
Configure the Kafka cluster
Monitor the data pipeline
Document the data pipeline

Career center

Learners who complete The Complete Apache Kafka Practical Guide will develop knowledge and skills that may be useful to these careers:

Reading list

We haven't picked any books for this reading list yet.

The Complete Apache Kafka Practical Guide

What's inside

Learning objectives

Syllabus

Good to know

Save this course

Activities

Career center

Reading list

Share

Similar courses