Implementing Software Architecture of a Large-Scale System from Udemy

The best way to learn software development is to write code. The best way to learn software architecture is to build and run a system. This is exactly what you are going to do as part of this course.

This course is the next in the series of Developer to Architect courses. The previous course 'Software Architecture & Technology of Large-Scale Systems' was focused on the concepts and principles of software architecture that govern modern large-scale systems.

This course 'Hands-On Implementation of Large-Scale Software Architecture', is focused on implementing those software and solution architecture principles in a mock large-scale system (mini system or simply minisys) by using open source technologies.

It starts with a downloadable code base of a mini system. We evolve this mini system into a full-fledged large-scale system implementation as we move along with the course. Our goal is to learn the typical architectural challenges that are faced by any large-scale system and solve these challenges using modern tools, technologies, and methods.

You can learn about these challenges and their solutions by actually working on a live system. For the same, you can download a mini system code-base provided by NewTechWays on Bitbucket or Github. You can follow the instructions to download, build and run this codebase. Although the instructions and code automation is platform-agnostic, this course will make use of Google Cloud Platform using a Linux machine as our test environment for building and running the system. Later we will put our system in a Production kind of environment using Kubernetes on Google Cloud Platform.

As we move along with this course we will cover the following architectural challenges and use the following technologies as their solutions:

3 Tier Application
- Webapp – Python Django
- RESTful Service – Java Spring Boot
- Database – PostgreSQL & Cassandra
- SPA - ReactJS
Containerization of System
- Docker Containers
- Docker Compose
Request Routing in Microservices
- Discovery – Netflix Eureka
- Gateway – Netflix Zuul
- Client-Side Load Balancer – Ribbon
- Server-Side Load Balancer - Nginx
System Observability
- Logging – Elasticsearch, Fluent, Kibana
- Tracing - Jaeger
- Monitoring & Alerts – Prometheus
Large-scale Data Management
- Caching - Redis
- Asynchronous Processing - RabbitMQ
- Horizontal Data Partitioning - Cassandra
Production Deployment
- Cloud Deployment – Kubernetes on GCP
- Auto-scaling – Kubernetes
- High Availability - Kubernetes
- Rolling Upgrades - Kubernetes

What's inside

Learning objectives

Understand through code, the architectural challenges of developing large-scale systems - scalability, reliability, deployment, observability
Code implementation of various tools and technologies used for making large-scale systems - redis, rabbitmq, netflix, fluentd, elasticsearch, kubernetes & more

Downloadable code along with configuration, scripts and instructions to run a mini system on your laptop and on a cloud environment
A practical demonstration of how systems are scaled and operated on both on-premises environment and in cloud environments

Understand through code, the architectural challenges of developing large-scale systems - scalability, reliability, deployment, observability
Code implementation of various tools and technologies used for making large-scale systems - redis, rabbitmq, netflix, fluentd, elasticsearch, kubernetes & more
Downloadable code along with configuration, scripts and instructions to run a mini system on your laptop and on a cloud environment
A practical demonstration of how systems are scaled and operated on both on-premises environment and in cloud environments

Syllabus

Course Overview

Introduction

Course Content Pdf

Work Environment Setup

Work environment

Create Linux workstation VM on cloud

Build code

Workstation VM backup

System components architecture, programming language, framework choices and code overview

System components overview

Running the system on cloud Linux VM

Web app architecture and technology selection

Python Django web app code overview

Services architecture

Services technology selection

Java SpringBoot services code overview

Database technology selection

Postgres RDBMS schema overview

Single page app technology selection

ReactJS SPA architecture

How to containerise and run a large number of system components as containers

Why containerize application components?

System with docker containers

Dockerize django webapp

Docker compose for automation

Dockerize postgres database

Dockerize java springboot services

Dockerize reactjs single page app

Running all dockerized components

Running system with docker network

Docker network and volumes

System review with docker containers

Solving scale related challenges with Microservices

Client routing challenge & Gateway services

Netfix Zuul gateway service code and configuration

Running system with Netfliz Zuul gateway service

Static routing challenge & Discovery services

Netflix Eureka discovery service code and configuration

Running system with Netflix Eureka discovery service

Client based & Server based Load balancing

Netflix Ribbon load balancer code and configuration

Running system with Nginx & Ribbon load balancers

Importance of centralized logging and implementation with EFK stack

System Observability - Logging, Tracing and Monitoring

Logging challenges in large-scale systems

Centralized logging architecture

Incrementing VM capacity

Configuration of Elasticsearch, Fluentd & Kibana (EFK stack)

Observing system with Elasticsearch, Fluentd, Kibana stack

Importance of distributed tracing and implementation with Jaeger

Tracing challenges in large-scale systems

Distributed tracing architecture

Distributed tracing with Uber Jaeger

Distributed tracing core mechanism

Uber Jaeger instrumentation code and configuration

Observing API latency using Jaeger tracing

Importance of metrics monitoring and implementation with Prometheus

Metrics for large-scale systems

Exposing application component metrics for Prometheus

Prometheus configuration for metrics

Observing system metrics through Prometheus

Data Caching with Redis Cache

Challenges of high request and data volume

Caching for managing read only load

Caching technology selection

Service code changes for Redis cache

Redis cache configuration

Running system with Redis cache

Asynchronous Processing with RabbitMQ

Managing write only load for Async workflows

Messaging queue technology selection

Service code changes for RabbitMQ

RabbitMQ configuration

Running system with RabbitMQ

Importance of partitioning of data and multi node writes

RDBMS Limitations and Data Partitioning

Distributed database technology selection

Cassandra schema vs Postgres schema

Service code changes for Cassandra

Cassandra configuration

Running system with highly available Cassandra database cluster

Deployment with Kubernetes on Google Cloud

Moving from test deployment to production deployment

Reasons for Kubernetes and Cloud deployment

Kubernetes cluster architecture

Setting up Cloud Container Registry

Creating Kubernetes Cluster on Google Cloud

Kubernetes configuration for the system

Deploying system on Kubernetes

Exposing system to Internet using External Load Balancers

High availability with Kubernetes

Auto-scaling with Kubernetes

Rolling upgrades with Kubernetes

Shutting down Kubernetes cluster

Good to know

Know what's good

, what to watch for

, and possible dealbreakers

Provides hands-on experience implementing software architecture principles, which is invaluable for developers looking to transition into architecture roles

Uses a practical approach by evolving a mini-system into a large-scale system, which mirrors real-world development scenarios and challenges

Covers a wide range of technologies including Python Django, Java Spring Boot, ReactJS, PostgreSQL, Cassandra, Docker, Kubernetes, and more, which are highly relevant in modern software development

Emphasizes system observability with tools like Elasticsearch, Fluentd, Kibana, Jaeger, and Prometheus, which are crucial for monitoring and maintaining large-scale systems

Requires familiarity with the previous course 'Software Architecture & Technology of Large-Scale Systems', which may be a barrier for learners without prior knowledge of software architecture concepts

Relies on Google Cloud Platform and a Linux machine as the test environment, which may require learners to have some familiarity with cloud environments and Linux-based systems

Activities

Be better prepared before your course. Deepen your understanding during and after it. Supplement your coursework and achieve mastery of the topics covered in Implementing Software Architecture of a Large-Scale System with these activities:

Brush up on Docker fundamentals

Show steps

Reviewing Docker concepts will help you understand the containerization aspects of the course.

Browse courses on Docker

Show steps

Complete a Docker tutorial.
Practice building and running simple containers.

Review Spring Boot basics

Show steps

Refreshing your knowledge of Spring Boot will make it easier to follow the service implementation sections.

Browse courses on Spring Boot

Show steps

Read the Spring Boot documentation.
Build a simple REST API with Spring Boot.

Read 'Building Microservices' by Sam Newman

Show steps

Reading this book will provide a deeper understanding of the microservices concepts covered in the course.

View Building Microservices: Designing Fine-Grained... on Amazon

Show steps

Read the book chapter by chapter.
Take notes on key concepts and examples.

Five other activities

Expand to see all activities and additional details

Show all eight activities

Implement a simple microservice

Show steps

Building a microservice will help you solidify your understanding of the concepts taught in the course.

Browse courses on Microservices

Show steps

Design a simple API.
Implement the API using Spring Boot or a similar framework.
Deploy the microservice to a container.

Document the architecture of the mini system

Show steps

Documenting the architecture will force you to deeply understand the system's design and components.

Browse courses on Software Architecture

Show steps

Explore the mini system code base.
Create diagrams illustrating the system architecture.
Write a document describing the components and their interactions.

Read 'Kubernetes in Action' by Marko Lukša

Show steps

Reading this book will provide a deeper understanding of Kubernetes and its role in deploying large-scale systems.

View Kubernetes in Action, Second Edition on Amazon

Show steps

Read the book chapter by chapter.
Experiment with the examples provided in the book.

Create a monitoring dashboard

Show steps

Creating a monitoring dashboard will help you understand how to observe and manage a large-scale system.

Browse courses on Prometheus

Show steps

Set up Prometheus to collect metrics from the mini system.
Configure Grafana to visualize the metrics.
Create a dashboard that shows key performance indicators.

Contribute to an open-source project

Show steps

Contributing to an open-source project will give you practical experience working on a real-world system.

Browse courses on Open Source

Show steps

Find an open-source project that uses the technologies covered in the course.
Identify a bug or feature to work on.
Submit a pull request with your changes.

Career center

Learners who complete Implementing Software Architecture of a Large-Scale System will develop knowledge and skills that may be useful to these careers:

Software Architect

A Software Architect designs the structure of software systems, ensuring they meet functional and non-functional requirements. This course directly aligns with that role, as it emphasizes the hands-on implementation of software architecture principles in a mock large-scale system. The course explores architectural challenges and their solutions using modern tools, technologies, and methods, which is precisely what a Software Architect does. The practical experience of working with a live system, containerizing applications with Docker, dealing with microservices, and implementing distributed systems using Kubernetes, all build toward the skills one needs to be an effective Software Architect.

See salaries and explore the career path for Software Architect

Cloud Solutions Architect

Cloud Solutions Architects design and implement cloud-based solutions. This course provides practical skills necessary to succeed in that role by working with Google Cloud Platform, demonstrating cloud deployment with Kubernetes, and highlighting auto-scaling and high-availability principles. The hands-on experience of building and running a system in the cloud, combined with the exploration of technologies such as Docker, Elasticsearch, and Prometheus, makes this course highly relevant for anyone looking to become a Cloud Solutions Architect. The focus on production deployment using Kubernetes further enhances a learner's readiness for the field.

See salaries and explore the career path for Cloud Solutions Architect

DevOps Engineer

DevOps Engineers focus on automating and streamlining software development and deployment processes. This course is exceptionally useful for this role because it provides hands-on experience with containerization using Docker, managing microservices, setting up continuous integration/continuous deployment pipelines with Kubernetes, and implementing system observability with tools like Elasticsearch, Fluentd, and Kibana. Practical experience with the deployment of tools, services, and databases is extremely beneficial for a DevOps Engineer, learning how to manage large scale systems efficiently. The course provides skills to manage systems on both on-premise and cloud environments.

See salaries and explore the career path for DevOps Engineer

Systems Engineer

Systems Engineers work on the overall architecture and performance of computer systems. This course provides a hands-on approach, allowing participants to understand system components, architecture, and technology choices. It covers challenges related to scaling, reliability, and observability, which are core concerns for a Systems Engineer. The course's focus on containers, Kubernetes, and monitoring tools like Prometheus and Jaeger provides practical skills to resolve these issues. The process of building and running a live system from the code level is paramount and necessary knowledge for a Systems Engineer.

See salaries and explore the career path for Systems Engineer

Backend Developer

Backend Developers build and manage the server-side logic of applications. This course is very useful since it deals directly with the technologies that a backend developer uses, especially those working with large, distributed systems. The course involves developing RESTful services with Java Spring Boot, using databases like PostgreSQL and Cassandra, and implementing asynchronous processing with RabbitMQ. The hands-on exploration of containerization, microservices, and data management is invaluable for any Backend Developer. The course helps build a foundation for understanding, building, and maintaining the back-end of large-scale systems.

See salaries and explore the career path for Backend Developer

Software Engineer

Software Engineers develop, test, and maintain software systems. This course offers practical experience in the full software development lifecycle, from initial code through deployment and monitoring. The course helps improve a Software Engineer's skill set with hands-on implementation of a mini system and addressing architectural challenges. The course covers a wide array of technologies like Python, Java, React, Docker, and Kubernetes are essential for a Software Engineer, especially for those involved in the development of large-scale systems. A course like this is crucial for any Software Engineer seeking practical, hands-on knowledge.

See salaries and explore the career path for Software Engineer

Solutions Engineer

Solutions Engineers work to understand and solve the technical challenges that customers face, which often includes implementing and deploying software. This course may help a Solutions Engineer, by providing in-depth, practical knowledge of building and deploying large-scale systems. The course covers a number of technologies that a Solutions Engineer would assist customers with, like Docker, Kubernetes, and cloud deployment on Google Cloud Platform. The hands-on approach to implementing all of these technologies, would help them more effectively communicate their value to end users.

See salaries and explore the career path for Solutions Engineer

Data Engineer

Data Engineers design, build, and maintain data pipelines and infrastructure. This course may be useful for a Data Engineer, by demonstrating a variety of data management techniques, particularly those relevant to large-scale systems. Data Engineers will learn how to manage data with Cassandra, how to cache with Redis, and how to implement asynchronous processing with RabbitMQ. This practical, hands-on experience with these technologies is valuable for any Data Engineer who needs to manage large amounts of data or needs to process streams of data efficiently.

See salaries and explore the career path for Data Engineer

Technical Lead

Technical Leads are responsible for guiding development teams and making key technical decisions. This course may be useful for a Technical Lead as it covers a wide range of technologies used in modern large-scale systems. The course provides practical experience in implementing architectural principles, dealing with microservices, and deploying systems using Kubernetes, which enhances their perspective. This course can help a Technical Lead to better understand and manage the teams they oversee. Working with a live system will deepen their knowledge.

See salaries and explore the career path for Technical Lead

Database Administrator

Database Administrators maintain and optimize database systems. This course may be useful for a Database Administrator because it explores database technologies like PostgreSQL and Cassandra, as well as caching with Redis. The course teaches about database schema, partitioning, and running highly available, distributed data clusters. While the course doesn't focus solely on administration, the hands-on experience with these databases is beneficial for a Database Administrator who wants to expand their skillset beyond just traditional relational databases.

See salaries and explore the career path for Database Administrator

Network Engineer

Network Engineers design and maintain computer networks. This course may be useful for a Network Engineer who wants to learn more about the technologies used in modern, large-scale software systems. The course introduces concepts such as request routing in microservices, load balancing, and service discovery. Additionally, the use of Kubernetes for deployment and networking provides relevant context. While not directly focused on networking, the course's coverage of network and distributed systems concepts would be a valuable complement to any Network Engineer's skillset.

See salaries and explore the career path for Network Engineer

IT Manager

IT Managers oversee an organization's technology infrastructure and operations. This course may be useful for an IT Manager, even if they aren't hands-on. The course provides a practical understanding of the technologies that form modern large-scale systems, such as cloud deployment, Kubernetes, and containerization with Docker. This general understanding will aid them in making better decisions regarding their organization's infrastructure. The course may not make them a technical expert, but will improve their understanding and insight into the systems their team runs.

See salaries and explore the career path for IT Manager

Technical Project Manager

Technical Project Managers oversee and coordinate technology projects. This course may be useful to a Technical Project Manager, as it exposes them to the various challenges and technologies involved in developing large-scale systems. Although not directly relevant for day-to-day tasks, the practical experience with deployment, containerization, and other technologies will give them an understanding of the technical challenges that their teams are facing. This insight can help them better manage technical deadlines, and effectively communicate with team members.

See salaries and explore the career path for Technical Project Manager

Quality Assurance Engineer

Quality Assurance Engineers are responsible for testing software and ensuring its quality. This course may be useful to a Quality Assurance Engineer interested in understanding the complexities of large-scale systems, and how to test their performance. Testing large systems requires a broad and deep understanding of the components and architectural choices that go into their creation. This course would help a Quality Assurance Engineer gain that understanding by focusing on the hands-on aspects of implementation.

See salaries and explore the career path for Quality Assurance Engineer

Business Analyst

Business Analysts analyze processes, systems, and data to improve organizational efficiency. This course may still be useful for a Business Analyst who works in a tech-heavy environment, especially if they want to better understand the technical challenges that their teams address. Though this course is highly technical, understanding all of the technologies that go into building a complex system will allow them to communicate more effectively with other members of the organization. The practical skills learned here may help them with this endeavor.

See salaries and explore the career path for Business Analyst

Implementing Software Architecture of a Large-Scale System

What's inside

Learning objectives

Syllabus

Good to know

Save this course

Activities

Career center

Reading list

Share

Similar courses