Event-Driven Microservices: Spring Boot, Kafka and Elastic from Udemy

Hi there. My name is Ali Gelenler. I'm here to help you learn event-driven microservices architecture by applying best practices for real-life challenges.

Important Update: This course uses Twitter api as the source data. Twitter is planning to change the twitter developer account to a paid subscription. However this will not effect the course because I have already added a Mock Tweet implementation so that you will not need to create a Twitter developer account if it is turned into a paid service.

In this course, you will focus on the development of microservices. With the help of microservices you can independently develop and deploy your application components . You can also easily scale services according to each service's own resource needs, for example you can scale better and create more instances of a service that requires more requests.

You can always use the latest versions for spring boot, spring cloud and other dependencies in this course. Please just follow the last section's lectures to see the required code and configuration changes for updated versions. Also if you would like to use subtitles during the course, you can turn on the captions on videos as all lectures are updated with hand-written subtitles. I suggest using subtitles to make it easier to follow the lectures.

When moving from a monolith application to microservices architecture, some challenges will arise as a result of having a distributed application and system. In this course you will learn how to deal with these challenges using event-driven architecture (EDA) architecture with Apache Kafka.

With an event-driven architecture;

You will truly decouple the services and create resilient services because a service has no direct communication with other services
You will use asynchronous/non-blocking communication between services
You will use an event/state store (Kafka), and remove the state from the services for better scalability
Tanima: "This is one of the best course i ever had in udemy, instructor is super responsive and always deals with complex problem during the course, Thank you so much Professor i will always be grateful to you for this course, and will keep eye on your next course release."

You will develop a microservice architecture from scratch using the most recent software platforms, technologies, libraries and tools, following best practices, applying microservices patterns and using Java, Spring boot, Spring cloud, Spring Security, Kafka and Elasticsearch. We will also cover Event sourcing and Event-driven services using Kafka as the event store.

The microservices patterns that you will be implementing are:

Externalized configuration with Spring Cloud Config
CQRS with Kafka and Elastic search
Api versioning for versioning of Rest APIs
Service Registration and Discovery with Spring Cloud and Netflix Eureka
Api Gateway with Spring Cloud Gateway
Circuit breaker with Spring Cloud Gateway and Resilience4j
Rate limiting with Spring Cloud Gateway and Redis to use Redis as the Rate limiter
Distributed tracing with0 and OpenID connect protocols for Authentication and Authorization using Keycloak and JWT. The use of Oauth for authorization of services and OpenID connect for authentication is widely used in microservices archictecture with Spring boot security.
- Oliver Michels: "Amazing course that covers a lot of ground, i.e. Spring Boot, Kafka, Elastic, OAuth/Keycloak, etc. and shows how all those pieces fit together in a nice microservice architecture."
Another important topic that you will focus on is the monitoring of microservices. You will use Spring boot actuator and Prometheus with Micrometer which can be used to get metrics such as health, number of running threads, number of requests, response time and memory usage. You will also learn to use Grafana to create a nice dashboard to visualize the metrics that you obtained using Spring boot actuator and Prometheus.
While introducing event-driven microservices, you will understand the basics of Apache Kafka by covering Kafka topics, Kafka partitions, Kafka consumer and producer APIs, Kafka admin client and Avro messaging.
- Emre Demir: "This is not only a software tutorial. It is an advanced computer engineering course. The examples and descriptions are excellent. Full-stack and Back-end developers must take."
To communicate between microservices, apart from using Kafka messaging, I will also introduce Kafka Streams and show how to use Kafka Streams to create a temporary state store with the accumulated streaming data. This temporary data source will then be consumed by another microservice.
To communicate with the data stores in microservices I will use different Spring Data dependencies, such as Spring Data JPA with PostgreSQL, Spring Data Elasticsearch and Spring Kafka.
For the Api documentation, I will use Open Api 3 specification which implements Swagger 3.0. Open Api will create a RESTful interface for an API and help you to easily develop and consume a Rest API.
You will also learn how to use Hateoas (Hypermedia as the Engine of Application State) in a Rest Api with Spring Hateoas. Hateoas will provide links to resources on the server, and update the REST API response with these links to interact with the client dynamically. This will enable developing level 3 Rest APIs.
- Sidhartha Mallik: "This course is to the point - and great one for every level, strongly recommend"
There is also a reactive development section in this course which demonstrates querying elasticsearch reactively with Reactive Spring, WebFlux, WebClient and Reactive Thymeleaf. In this section you will learn how to use Spring Webflux and Spring reactive components to create an asynchronous flow between microservices.
You will use Spring boot and Spring cloud along with Docker for developing cloud ready microservices. Docker is a containerization platform and it enables to package applications into containers. These containers can later run in any environment that docker is running. In this course you will create the docker images of the microservices that you have developed. Then use docker compose to run all the containers of microservices together, along with the other dependencies such as Kafka, Elasticsearch, Keycloak, Redis, Prometheus, Grafana and Zipkin . In the end a single 'docker-compose up' command will be enough to run all microservices architecture together with the required dependencies. Having these docker images will enable you to deploy your application to any cloud vendor easily.
- Mohammed O. Tillawy: "amazing course, but prepare yourself, very intense, and tons of info, needs focus and patience."
You will be following a hands-on approach and be developing a project from scratch in which you will have multiple microservices surrounded with multiple modules to accomplish the specific tasks. So you will need to make your hands dirty in this course where I will be guiding you from start to finish. You will also find multiple choice quizes in each section to check your progress throughout the course.
At the end of the course you will not only understand the real life challenges of a distributed application with multiple services and components, but also you will be able to apply solutions to this challenges.
You can download the source code in each lecture separately and run them easily using your IDE or docker.
- Ido Charar: "This course by Ali Gelenler is outstanding. It is not just about Spring Cloud with kafka and elastic. This course gives you much more knowledge around technologies related to Spring Cloud in particular and Cloud Computing in general. Among them are technologies like reactive programming, streaming, linux OS, security, Design Patterns and much much more. All the information is given in succinct but ingestable form, which allows you broaden your skills in shortest possible time. Very appreciate the investment in material, lecturer involvement, constant assistance and help to the students. Will recommend to take this course everyone who want to level up her/his skills."
For more detailed information on the progress of this course, you can check the introductory video and free lessons, and if you decide to enroll in this course, you are always welcome to ask and discuss the concepts and implementation details on Q/A and messages sections. I will guide you from start to finish to help you successfully complete the course and gain as much knowledge and experience as possible from this course.
- Shantanu Srivastava: "I brought this course specifically for learning Apache Kafka and after going through the Introduction video, I find out that this course will cover literally every single thing you need to know as a spring developer. From API gateways and Service Discovery, to log forwarding and resilience and so much more. Awesome. "
Remember. There is a 30-day full money-back guarantee for this course. So you can safely press the 'Buy this course' button with zero risk and join this learning journey with me.

What's inside

Learning objectives

Master microservices architecture with spring boot and spring cloud stack
Develop event-driven microservices with apache kafka
Understand containerization of microservices with docker and docker compose
Learn basics of kafka brokers, topics, partitions, producers, consumers, admin client, avro messaging, zookeper and schema registry
Learn basics of elasticsearch, index api and query api
Implement kafka based messaging applications with spring kafka
Understand elasticsearch indexing and queries with spring data elasticsearch
Implement cqrs pattern with kafka and elasticsearch
Implement externalize configuration pattern with spring cloud config
Develop rest api
Implement api versioning for versioning of rest apis
Implement spring data jpa with postgresql
Understand service registration and discovery with netflix eureka and spring cloud
Implement client side load balancing with spring cloud load balancer
Understand api gateway pattern with spring cloud gateway

Implement circuit breaker pattern with spring cloud gateway and resilience4j
Implement rate limiting with spring cloud gateway and redis
Understand authentication and authorization with spring security oauth 2.0 and openid connect protocols with keycloak using jwt
Implement monitoring of microservices with spring boot actuator, micrometer, prometheus and grafana
Understand log aggregation patterns and visualization with elk stack - elasticsearch, logstash and kibana
Understand using hateoas in rest apis with spring hateoas
Implement open api v3 for restful api documentation with swagger v3
Use thymeleaf and bootstrap for a basic front-end
Understand reactive development and asynchronous flow by querying elasticsearch reactively with reactive spring, webflux, webclient and reactive thymeleaf
Learn distributed tracing with slf4j mdc, spring cloud sleuth and zipkin
Understand database per service pattern
Understand using kafka streams with state store
Learn streaming tweets from twitter using twitter4j
Show more
Show less

Master microservices architecture with spring boot and spring cloud stack
Develop event-driven microservices with apache kafka
Understand containerization of microservices with docker and docker compose
Learn basics of kafka brokers, topics, partitions, producers, consumers, admin client, avro messaging, zookeper and schema registry
Learn basics of elasticsearch, index api and query api
Implement kafka based messaging applications with spring kafka
Understand elasticsearch indexing and queries with spring data elasticsearch
Implement cqrs pattern with kafka and elasticsearch
Implement externalize configuration pattern with spring cloud config
Develop rest api
Implement api versioning for versioning of rest apis
Implement spring data jpa with postgresql
Understand service registration and discovery with netflix eureka and spring cloud
Implement client side load balancing with spring cloud load balancer
Understand api gateway pattern with spring cloud gateway
Implement circuit breaker pattern with spring cloud gateway and resilience4j
Implement rate limiting with spring cloud gateway and redis
Understand authentication and authorization with spring security oauth 2.0 and openid connect protocols with keycloak using jwt
Implement monitoring of microservices with spring boot actuator, micrometer, prometheus and grafana
Understand log aggregation patterns and visualization with elk stack - elasticsearch, logstash and kibana
Understand using hateoas in rest apis with spring hateoas
Implement open api v3 for restful api documentation with swagger v3
Use thymeleaf and bootstrap for a basic front-end
Understand reactive development and asynchronous flow by querying elasticsearch reactively with reactive spring, webflux, webclient and reactive thymeleaf
Learn distributed tracing with slf4j mdc, spring cloud sleuth and zipkin
Understand database per service pattern
Understand using kafka streams with state store
Learn streaming tweets from twitter using twitter4j
Show more
Show less

Syllabus

Introduction to microservices architecture

Main objectives and benefits of the course.

The architectural big picture of the project and introduction to event-driven architecture and event-driven services.

The implementation details of all microservices and components that the course covers.

Microservices description

How to set up the development environment

Installation instructions of the tools required throughout the course.

Learn to create a Spring boot Microservice that reads data from Twitter and writes to Kafka. Learn using Kafka as an event source, Kafka producers and Twitter4j library

Introduction to spring boot, starter modules and pre-configured dependencies.

Create a base project using start.spring.io. Adjust the pom.xml file to prepare it as a base file for sub-modules. Explain the basics of maven dependencies and plugins.

Which one of options below cannot be used with spring-boot-maven-plugin?

The first microservice as a maven module. Create a spring boot application. Create a basic test class. Implement initialization logic using CommandLineRunner and explain other options. Add configuration variables using application.yml. Add logging using slf4j and a logback.xml file. Describe constructor injection, and understand advantages over field injection.

Understanding the log levels

Advantages of constructor injection over field injection

Streaming tweets with Twitter4j: The command component in CQRS & Event sourcing

Streaming tweets with Twitter Api V2

Learn annotations that spring scans to create a bean

Adding mock twitter stream as an alternative

Introducing Apache Kafka:Event sourcing, topics, partitions, producer & consumer

Describe kafka consumers

Adding common config module

Running Apache Kafka cluster with docker: Kafka, Zookeeper and Schema Registry

Creating kafka-model module

Creating kafka-admin module - Part 1: Configuration and dependencies

Creating kafka-admin module - Part 2: Creating Kafka topics programmatically

Creating kafka-producer module: Configuration of Kafka producer

Understand the kafka producer configuration basics

Creating kafka-producer module: Produce events to store in Kafka event store

Understand the Kafka producer implementation basics

Integrate Kafka modules with Microservice: Use Kafka as event store for service

Containerization of microservice with docker image: Run all with docker compose

Learn to externalize configuration from the application source code using Spring cloud config server

Creating Config Server Repository

Creating Spring Cloud Config Server as a Microservice

Understand the steps required to enable a config server

Using a common logback file for all microservices

Changing twitter-to-kafka-service to work with config server

Using remote GitHub repository

Adding security to config server and encrypt passwords

Using Jasypt to encrypt sensitive data

Using JCE to encrypt sensitive data

JCE vs Jasypt

Understand PBKDF2

Containerization of config server by creating the docker image

Create a Spring boot Microservice that reads data from Kafka and writes to Elasticsearch. Learn Kafka consumers and Elasticsearch index api

Introduction to Kafka Consumer: Reading data events from Kafka using messaging

Adding kafka-consumer module

Understand Kafka consumer properties

Understanding Kafka consumer properties

Creating the microservice: kafka-to-elastic-service

Adding initialization check

Introducing Elasticsearch

Running elastic search with docker

Creating elastic-model module

Understanding elasticsearch Field annotation

Creating elastic-config module

Creating elastic-index-client module

Using Elasticsearch repositories for indexing

Understand the differences between ElasticsearchOperations and ElasticsearchRepository

Integrating elastic modules with microservice

Create a new microservice that will include Rest API, versioning, hateoas, Open Api Docs v3 and elasticsearch query capabilities. Learn different type of elastic queries.

Introduction to Elastic Query Api

Understand elasticsearch query types

Creating elastic-query-client module

Using spring data elasticsearch repository for querying

Understand elasticsearch client basics

Elastic query microservice: The query component in CQRS & Event sourcing

Creating web controller

Creating business layer to return elastic data in controller

Creating validation and controller advice

Introduction to Hateoas (Hypermedia as the Engine of Application State)

Adding Hateoas capability to Rest API with Spring Hateoas

Introduction to Api versioning of Rest API

Understand types of api versioning

Implementing Api versioning for elastic query service Rest API

Enabling Open Api v3 for documentation of RESTful API with Swagger v3

Containerization of microservice by creating the docker image

Create a new web client as a spring boot service that will include REST Api, webclient, thymeleaf, bootstrap.

Introduction to Web clients

Starting to implement elastic-query-web-client

Creating Thymeleaf templates with Bootstrap

Creating Rest API controller

Understand using controller with thymeleaf

Creating configurations for web client and security

Creating Webclient implementation

Adding client side load balancer with Spring Cloud Load Balancer

Understand how Client Side Load Balancer works

Learn reactive development and querying Elasticsearch reactively with Reactive Spring Data Elasticsearch, Reactive Spring WebClient, WebFlux and reactive Thymeleaf

Creating common service and web client modules to re-use them with DRY principle

Creating reactive elastic query service: Reactive Spring and Flux reactive type

Understand using reactive Rest controller

Creating reactive elastic web client: Reactive Spring, WebFlux and WebClient

Learn implementing Authentication and Authorization with Oauth2 and OpenID connect protocols with Keycloak authz server and JWT

Introduction to Oauth and Keycloak authorization server

Understanding Oauth 2.0 protocol flows

Run and configure Keycloak authorization server with OIDC for authentication

Understanding Spring security with Spring Security OAuth 2.0 resource server

Configuring query service with Spring security Oauth 2.0, OpenId Connect and JWT

Good to know

Know what's good

, what to watch for

, and possible dealbreakers

Perfects for long-time Java users who want to transition into the realm of Microservices

In-depth and thorough coverage of microservices

Provides hands-on practice and examples to solidify understanding

Regular updates keep the course content fresh and relevant to industry standards

May require some prior experience with Java

Activities

Be better prepared before your course. Deepen your understanding during and after it. Supplement your coursework and achieve mastery of the topics covered in Event-Driven Microservices: Spring Boot, Kafka and Elastic with these activities:

Step-by-Step Spring Kafka Tutorial

Show steps

Follow a guided tutorial to integrate Spring Kafka into your Java applications, covering concepts such as message production, consumption, and error handling.

Browse courses on Spring Boot

Show steps

Set up a Spring Boot project.
Add Spring Kafka dependencies to your project.
Create a Kafka producer and configure it using Spring Kafka annotations.
Create a Kafka consumer and configure it using Spring Kafka annotations.

Read 'Kafka: The Definitive Guide'

Show steps

Comprehend Kafka's architecture, design principles, and best practices by reading the authoritative book 'Kafka: The Definitive Guide', providing a solid foundation for working with Kafka.

View Kafka: The Definitive Guide: Real-Time Data and... on Amazon

Show steps

Acquire a copy of 'Kafka: The Definitive Guide'.
Read and understand the chapters relevant to your learning objectives.
Apply the knowledge gained to your Kafka projects.

Review Apache Kafka Components

Show steps

Review the main components and concepts of Kafka, such as producers, consumers, topics, and partitions to strengthen your understanding of Kafka's architecture.

Browse courses on Apache Kafka

Show steps

Read Apache Kafka documentation.
Set up a local Kafka cluster for testing and experimentation.
Practice writing Kafka producers and consumers.

Four other activities

Expand to see all activities and additional details

Show all seven activities

Compile Kafka Resources and Tools

Show steps

Build a collection of valuable Kafka resources, including documentation, tutorials, and tools, to serve as a reference and support your Kafka learning journey.

Browse courses on Apache Kafka

Show steps

Search and gather relevant Kafka resources from reliable sources.
Organize and categorize the resources for easy access.
Keep the compilation updated with the latest resources and tools.

Kafka Command-Line Exercises

Show steps

Gain practical experience with Kafka by using the command-line interface (CLI) to perform common tasks, such as creating topics, publishing messages, and consuming messages.

Browse courses on Apache Kafka

Show steps

Install the Kafka CLI tools.
Create a Kafka topic using the CLI.
Produce messages to the topic using the CLI.
Consume messages from the topic using the CLI.

Participate in Kafka Hands-on Workshop

Show steps

Enroll in a hands-on Kafka workshop to gain in-depth knowledge and practical experience in working with Kafka, covering topics such as configuration, performance tuning, and security.

Browse courses on Apache Kafka

Show steps

Find a reputable Kafka workshop provider.
Register for the workshop and prepare for the hands-on exercises.
Actively participate in the workshop and complete all exercises.

Develop a Kafka-Based Microservice

Show steps

Build a real-world microservice that utilizes Kafka for event-driven communication, allowing you to apply your knowledge and gain hands-on experience in designing and implementing microservices.

Browse courses on Microservices

Show steps

Design the microservice architecture.
Implement the Kafka producer and consumer components.
Deploy the microservice and test its functionality.

Career center

Learners who complete Event-Driven Microservices: Spring Boot, Kafka and Elastic will develop knowledge and skills that may be useful to these careers:

Microservices Developer

A Microservices Developer designs, develops, and maintains microservices. This course provides a comprehensive overview of microservices, including how to use Spring Boot, Spring Cloud, and Docker to build and deploy them. The course also covers best practices for monitoring and securing microservices. Taking this course can help you gain the skills and knowledge you need to succeed as a Microservices Developer.

See salaries and explore the career path for Microservices Developer

Event-Driven Microservices Architect

An Event-Driven Microservices Architect designs and implements microservices that use event-driven architecture to communicate with each other. This course provides a comprehensive overview of event-driven microservices, including how to use Apache Kafka, Spring Boot, and Spring Cloud to build and deploy them. The course also covers best practices for monitoring and securing microservices. Taking this course can help you gain the skills and knowledge you need to succeed as an Event-Driven Microservices Architect.

See salaries and explore the career path for Event-Driven Microservices Architect

Software Architect

A Software Architect designs and implements software systems. This course provides a comprehensive overview of software architecture, including how to use microservices, event-driven architecture, and DevOps to build and deploy software systems. The course also covers best practices for monitoring and securing software systems. Taking this course can help you gain the skills and knowledge you need to succeed as a Software Architect.

See salaries and explore the career path for Software Architect

Data Engineer

A Data Engineer designs and implements data pipelines. This course provides a comprehensive overview of data engineering, including how to use Apache Kafka, Spark, and Hadoop to build and deploy data pipelines. The course also covers best practices for monitoring and securing data pipelines. Taking this course can help you gain the skills and knowledge you need to succeed as a Data Engineer.

See salaries and explore the career path for Data Engineer

Data Scientist

A Data Scientist uses data to solve business problems. This course provides a comprehensive overview of data science, including how to use Python, R, and machine learning to build and deploy data science solutions. The course also covers best practices for monitoring and securing data science solutions. Taking this course can help you gain the skills and knowledge you need to succeed as a Data Scientist.

See salaries and explore the career path for Data Scientist

Machine Learning Engineer

A Machine Learning Engineer designs and implements machine learning solutions. This course provides a comprehensive overview of machine learning, including how to use Python, R, and machine learning algorithms to build and deploy machine learning solutions. The course also covers best practices for monitoring and securing machine learning solutions. Taking this course can help you gain the skills and knowledge you need to succeed as a Machine Learning Engineer.

See salaries and explore the career path for Machine Learning Engineer

Cloud Architect

A Cloud Architect designs and implements cloud-based solutions. This course provides a comprehensive overview of cloud computing, including how to use AWS, Azure, and GCP to build and deploy cloud-based solutions. The course also covers best practices for monitoring and securing cloud-based solutions. Taking this course can help you gain the skills and knowledge you need to succeed as a Cloud Architect.

See salaries and explore the career path for Cloud Architect

DevOps Engineer

A DevOps Engineer automates the process of building, testing, and deploying software systems. This course provides a comprehensive overview of DevOps, including how to use Docker, Jenkins, and Kubernetes to build and deploy software systems. The course also covers best practices for monitoring and securing software systems. Taking this course can help you gain the skills and knowledge you need to succeed as a DevOps Engineer.

See salaries and explore the career path for DevOps Engineer

Security Engineer

A Security Engineer designs and implements security solutions. This course provides a comprehensive overview of security, including how to use firewalls, intrusion detection systems, and security monitoring tools to build and deploy security solutions. The course also covers best practices for monitoring and securing security solutions. Taking this course can help you gain the skills and knowledge you need to succeed as a Security Engineer.

See salaries and explore the career path for Security Engineer

Data Analyst

A Data Analyst uses data to identify trends and patterns. This course provides a comprehensive overview of data analysis, including how to use Python, R, and SQL to analyze data. The course also covers best practices for monitoring and securing data analysis solutions. Taking this course can help you gain the skills and knowledge you need to succeed as a Data Analyst.

See salaries and explore the career path for Data Analyst

IT Manager

An IT Manager plans and executes IT projects. This course provides a comprehensive overview of IT management, including how to use IT planning, IT budgeting, and IT project management to plan and execute IT projects. The course also covers best practices for monitoring and securing IT projects. Taking this course can help you gain the skills and knowledge you need to succeed as an IT Manager.

See salaries and explore the career path for IT Manager

Business Analyst

A Business Analyst analyzes business needs and provides solutions. This course provides a comprehensive overview of business analysis, including how to use data analysis, process mapping, and requirements gathering to analyze business needs and provide solutions. The course also covers best practices for monitoring and securing business analysis solutions. Taking this course can help you gain the skills and knowledge you need to succeed as a Business Analyst.

See salaries and explore the career path for Business Analyst

Full-Stack Developer

A Full Stack Developer designs and implements both the front-end and back-end of web applications. This course provides a comprehensive overview of full stack development, including how to use HTML, CSS, JavaScript, Java, and Spring Boot to build and deploy full stack web applications. The course also covers best practices for monitoring and securing full stack web applications. Taking this course can help you gain the skills and knowledge you need to succeed as a Full Stack Developer.

See salaries and explore the career path for Full-Stack Developer

Project Manager

A Project Manager plans and executes projects. This course provides a comprehensive overview of project management, including how to use project planning, project scheduling, and project budgeting to plan and execute projects. The course also covers best practices for monitoring and securing projects. Taking this course can help you gain the skills and knowledge you need to succeed as a Project Manager.

See salaries and explore the career path for Project Manager

Web Developer

A Web Developer designs and implements the front-end of web applications. This course provides a comprehensive overview of web development, including how to use HTML, CSS, JavaScript, and React to build and deploy web applications. The course also covers best practices for monitoring and securing web applications. Taking this course can help you gain the skills and knowledge you need to succeed as a Web Developer.

See salaries and explore the career path for Web Developer