This course is structured to give you a theoretical and coding experience with Apache Kafka using SpringBoot. This course is targeted for developers who would like to build enterprise standard Kafka Client applications using SpringBoot.
If you are looking forward to learning the below-listed things:
Use cases where Kafka fits really well
Internals of Kafka and how it works
Built Enterprise Standard Kafka Client Applications using Producer/Consumer API using Spring Boot
Unit/Integration Tests for the Kafka Client Applications
This course is structured to give you a theoretical and coding experience with Apache Kafka using SpringBoot. This course is targeted for developers who would like to build enterprise standard Kafka Client applications using SpringBoot.
If you are looking forward to learning the below-listed things:
Use cases where Kafka fits really well
Internals of Kafka and how it works
Built Enterprise Standard Kafka Client Applications using Producer/Consumer API using Spring Boot
Unit/Integration Tests for the Kafka Client Applications
Then this is the right course for you. This is a pure hands-on oriented course where you will be learning the concepts through code.
By the end of this course, you will have a complete understanding of coding and implementing Kafka Clients using SpringBoot with Producer/Consumer API.
Getting Started with Kafka
In this section, I will give you all a quick introduction to Apache Kafka, terminologies and different client APIs that are part of Kafka
Download and Install Kafka
In this section, we will download the Kafka distribution from the Kafka Website.
Understanding Kafka Components and its Internals - (Theory + Hands-On)
In this section, we will explore the Kafka internals from a theoretical perspective followed by hands-on exploring the Kafka internals.
Learn about the Topics and Partitions in Kafka
Setup a Local Kafka Cluster with Multiple Brokers
Producer/Consumer messages in the Kafka Cluster
Learn about Consumer Offsets and Consumer Groups
Commit Log and Retention Policy
Learn about Kafka Load Distribution and Fault Tolerance and Robustness
Application OverView
This section covers the application that we are going to build as part of this course.
Build SpringBoot Kafka Producer - Hands-On
In this section, we will learn about building a Kafka Producer using Spring Boot.
Build RestAPI through which the events can be posted into the application from the outside world
Explore KafkaTemplate to publish the data into the Kafka Topic
Learn different approaches to produce the message into the Kafka
Learn to publish the Kafka Record using Headers
Integration Testing using JUnit5 - Hands-On
In this section, we will learn about different approaches to writing Integration tests using Embedded Kafka.
Write Integration test cases to interact with the API using JUnit5
Write Integration test cases to interact with Embedded Kafka using JUnit5
Unit Testing using JUnit5- Hands-On
In this section, we will learn about different approaches to writing unit tests Kafka Producer.
Write Unit Tests to the controller layer using @WebMVC annotation and MockMVC
Add the validations to the request payload in the endpoint
Write Custom Error Handler for different response codes
Kafka Producer - Sending Message With Key - Hands-On
In this section, we will learn about sending the record to Kafka Topic with Key.
Kafka Producer - Important Configurations
In this section, we will learn about different key configurations to consider when it comes to reliable message delivery to Kafka
Build SpringBoot Kafka Consumer - Hands-On
In this section, we will learn about building a Kafka Consumer using Spring Boot.
Set up the base consumer project library-events consumer
Learn about different Spring Kafka Terminologies which are necessary to configure a Kafka Consumer
Learn about how to configure a Kafka Consumer using the @KafkListener Annotation
Learn about how "Spring Boot Auto Configuration works behind the scenes for Kafka Consumer"
Consumer Groups and Consumer Offset Management- Hands-On
In this section, we will code and learn about consumer groups and offset management.
Demonstration how Consumer Groups plays a vital role when it comes to scalable message consumption and consumer rebalance
Learn about the Default consumer offset management by Kafka
Learn about how consumer offset can be manually managed from the application
Demonstration of how to achieve scalable message consumption by increasing the concurrency level to greater than 1
Persisting Library Events in DB - Using H2 InMemory DataBase
In this section, we will code and learn about integrating the DB layer into the library-events-consumer using Spring JPA.
Configuring the H2 In-Memory DB
Create LibraryEvent and Book Entity
Build Service Layer to process LibraryEvent - ADD Event Type
Build Service Layer to process LibraryEvent - MODIFY Event Type
Integration Testing using Embedded Kafka - Kafka Consumer
In this section, we will code and learn to write the Integration tests for the Kafka Consumer.
Configure Embedded Kafka for Integration Tests
Write the Integration test for posting a "NEW" LibraryEvent
Write the Integration test for posting an "UPDATE" LibraryEvent
Integration Tests for Real Databases using TestContainers
Error Handling, Retry and Recovery - Kafka Consumers
In this section, we will learn about the different error handling techniques that are available for Kafka Consumer.
Custom Error Handler
Retry in Kafka Consumer
Retry SpecificExceptions using Custom RetryPolicy
Recovery in Kafka Consumer
Handling Recovery in Kafka Consumer
Error Handling, Retry/Recovery - Kafka Producer
Error Handling in Kafka Producer
Retry in Kafka Producer - Broker Not Available
Retry in Kafka Producer - Min.in.sync.replicas
Retain/Recover Failed Records in Kafka Producer
By the end of this you will have a complete understand and knowledge of building enterprise standard Kafka Consumers and Producers using Spring Boot with the Unit and Integration tests using EmbeddedKafka.
In this lecture, I will cover the course introduction and objectives of this course.
In this lecture, I will cover the prerequisites that are needed to enroll in this course.
In this lecture, I will explain about How Kafka fits in to the current software architecture and cover some of the examples where Kafka can be used.
In this lecture, I will explain about the Kafka Terminologies and different components that are part of Apache Kafka and some of its Client APIs.
In this lecture, we will learn in detail about the Kafka topics, partitions.
In this lecture, we will set up the Zookeeper and Kafka broker in local.
In this lecture, we will learn about creating a topic, produce messages, and consume messages from a Kafka Topic.
In this lecture, we will learn about producing and consuming messages with key using the command line interface.
In this lecture, we will learn about consumer offsets how it plays a major role in keeping a record of resuming processing the records from where it left off in the topic.
In this lecture, we will learn about consumer groups and how it plays a major role in scalable message consumption.
In this lecture, we will learn about the partitioned commit log and retention policy of any record that's written into Kafka.
In this lecture, I will explain in detail about how does Kafka behaves as a Streaming System.
In this lecture, we will set up a Kafka Cluster with 3 brokers to simulate a production like environment.
In this lecture, we will learn about how Kafka distributes the request between the available brokers.
In this lecture, we will learn about replication and Insync Replicas ISR.
In this lecture, I will explain the importance of minimum in-sync replica.
In this lecture, I will explain about the application that we are going to build as part of this course.
In this lecture, I will attach the source code for this course.
In this lecture, we will set up the spring boot project for the Library Event Spring Boot Kafka project.
In this lecture, we will build the domain object for the Library Event.
In this lecture, we will build the post endpoint to publish the Library Event.
In this lecture, we will learn about KafkaTemplate which is part of "spring-kafka" to produce Messages.
In this lecture, we will code and learn "how to configure" a KafkaTemplate using Spring Boot AutoConfiguration.
In this lecture, we will code and learn about "How the SpringBoot AutoConfiguration" works ?
In this lecture, we will autocreate the Kafka topic using the KafkaAdmin configuration.
In this lecture, we will code the LibraryEvents Producer and integrate KafkaTemplate to produce Messages in to the library-events topic asynchronously.
In this lecture, I will explain the behavior of the "POST" endpoint "/libraryevent"
In this lecture, we will code the LibraryEvents Producer and integrate KafkaTemplate to produce Messages in to the library-events topic synchronously.
In this lecture, we will code the kafkaTemplate to publish the ProducerRecord with KafkaHeaders.
In this lecture , I will give you all a quick introduction to Automated Tests.
In this lecture, we will write the integration test for the POST endpoint.
In this lecture, we will write the integration test for the POST endpoint
In this lecture, we will learn about the need for Embedded Kafka and how it helps with the Integration testing.
In this lecture, we will learn about how to spin up a Kafka Consumer in JUNIT and consume the published record.
In this lecture , I will give you all a quick introduction to Unit Tests.
In this lecture, we will code and learn how to write unit tests for the post endpoint using MockMVC.
In this lecture, we will code and learn about how to add validations to the request body.
In this lecture, we will code and learn about how to add Custom Error Messages to the HttpResponse.
In this lecture, we will build the put endpoint to publish the library events updates in to the Kafka topic.
In this lecture, I will highlight some of the key configurations to consider for reliable message delivery to Kafka.
In this lecture, we will code and learn how to override the default producer config values in Spring Boot using AutoConfiguration.
In this lecture, we will set up the base consumer project library-events consumer.
In this lecture, we will learn about different Spring Kafka Terminologies which are necessary to configure a Kafka Consumer.
In this lecture, we will code and learn about how to configure a Kafka Consumer using Spring Boot AutoConfiguration.
In this lecture, we will code and learn about how to configure a Kafka Consumer using the @KafkListener Annotation.
In this lecture, I will explain about how Spring Boot Auto Configuration works behind the scenes for Kafka Consumer
In this lecture, I will demonstrate how Consumer Groups plays a vital role when it comes to scalable message consumption and consumer rebalance.
In this lecture, we will code and learn about the Default consumer offset management by Kafka.
In this lecture, we will learn about how consumer offset can be manually managed from the application.
In this lecture, I will demonstrate how to achieve scalable message consumption by increasing the concurrency level to greater than 1.
In this lecture, we will configure the in-memory DB in library events consumer.
In this lecture, we will code the entity classes to map the Library and the Book Entity.
In this lecture, we will code the service layer to process the "ADD" LibraryEvent.
In this lecture, we will code the service layer to process the "MODIFY" LibraryEvent.
In this lecture, we will code and learn to set up the Integration test for the Kafka Consumer.
In this lecture, we will code the first integration test for the Library Events Consumer for the ADD LibraryEventType.
In this lecture, we will code the first integration test for the Library Events Consumer for the MODIFY LibraryEventType.
In this lecture, we will code and write the integration test for the Library Events Consumer for the MODIFY LibraryEventType with null LibraryEventId.
In this lecture, I will talk about writing the integration tests for the real databases.
In this lecture, we will code and learn about implementing the custom error handler and retry in Kafka Consumer.
In this lecture, we will code and learn about implementing the RetryListener to monitor the outcome of each retry attempt.
In this lecture, we will code and learn about retrying only specific exceptions.
In this lecture, we will code and learn about retry using the exponential backoff pattern.
In this lecture, we will learn about different recovery techniques that can be applied to the Kafka Consumer when the retries are exhausted.
In this lecture, we will code and learn about the technique to publish the failed message to the retry topic.
In this lecture, we will code and learn about the approach to reprocess the message
In this lecture, we will code and learn about the refactoring the integration test case for LibraryEvents Consumer.
In this lecture, we will code and learn about the technique to publish the failed message to the deadletter topic.
In this lecture, we will code and learn about the technique to save the failed message to the DB for later processing.
In this lecture, we will code and learn about the technique to retry the failed message thats saved in the DB.
In this lecture, we will discuss about the different error handling options available in Kafka Producer.
In this lecture we will learn about the options that are available to retry a failed record in Kafka Producer.
In this lecture, we will learn about the recovery options that are available from the Kafka Producer end.
In this lecture, we will learn about the technique to retain the record that failed because of the Kafka Cluster issue.
In this lecture, we will discuss about the need for Kafka Security and How SSL Works?
In this lecture, how enterprises request for SSL certificate from Certificate Authority(CA).
In this lecture, we will set up the SSL security in our local Kafka Cluster
In this lecture, we will code and learn about accessing the SSL secured topic from our spring boot application.
OpenCourser helps millions of learners each year. People visit us to learn workspace skills, ace their exams, and nurture their curiosity.
Our extensive catalog contains over 50,000 courses and twice as many books. Browse by search, by topic, or even by career interests. We'll match you to the right resources quickly.
Find this site helpful? Tell a friend about us.
We're supported by our community of learners. When you purchase or subscribe to courses and programs or purchase books, we may earn a commission from our partners.
Your purchases help us maintain our catalog and keep our servers humming without ads.
Thank you for supporting OpenCourser.