We are entering an era in software engineering where we rely on libraries and frameworks to do most of our work. While this is useful and save tremendous dev hours, it creates leaky abstractions that manifest in form of performance degradation, unexpected errors, 100% CPU , network and disk usage, hangs, latency and many other. It becomes difficult to articulate what goes wrong on the backend and how to fix it when engineers don't understand the basic building block of what they are interacting with.
We are entering an era in software engineering where we rely on libraries and frameworks to do most of our work. While this is useful and save tremendous dev hours, it creates leaky abstractions that manifest in form of performance degradation, unexpected errors, 100% CPU , network and disk usage, hangs, latency and many other. It becomes difficult to articulate what goes wrong on the backend and how to fix it when engineers don't understand the basic building block of what they are interacting with.
Network communication is one of the most critical pieces in backend engineering. And the TCP/IP Internet suite is the basic fundemntal building block of communications that link the frontend to the backend. Any protocol we use eventually gets encapsulated into either a TCP segment or UDP datagram. That packet is wrapped in an IP packet which is placed into a frame and then sent across the network. These layers are important to understand for backend engineers in order to optimize the stack, minimize latency, and increase throughput. What matters is the true understanding of these pieces and not just memorizing definitions and packet headers to pass an exam. If an engineer truly understood what is happening when their backend API receives an HTTP POST request for example, they will be astonished at how much is happening behind the scenes. They would completely change the way they approach backend API designs, they will try to select the best protocol for the job, and they will tune that protocol to its maximum potential. They will start asking questions about frameworks and libraries that nobody else does. It is those questions that will make them fully utilize those framework to their maximum potential.
This course is primarily designed for backend engineers who built applications, services or APIs and want to take their skillset to the next level. If you are a frontend engineer who are interested in the backend and have attempted to build backend apps before this course is also for you. If you are a network engineer who already know the basic fundamentals and want to start building backend applications effectively this course might help you bridge the gap
This course is not designed for those who want to pass network certification tests (.)
You can download the slides and codes from here. The slides will be updated so make sure to check back often.
Quick quiz to test the understanding of these fundamentals.
List of private ip addresses https://en.wikipedia.org/wiki/Private_network
RFC 1918 that defines the private IP addresses https://www.rfc-editor.org/rfc/pdfrfc/rfc1918.txt.pdf (also attached)
Now that you know IP, TCP and UDP. Any other protocol must be built on top of these three.
TLS or Transport Layer security is a protocol used to encrypt the communication between two hosts. This lecture describes an overview of the protocol. RFCs attached and can be retrieved here.
https://www.rfc-editor.org/rfc/pdfrfc/rfc8446.txt.pdf
https://www.rfc-editor.org/rfc/pdfrfc/rfc5246.txt.pdf
John Nagle developed this algorithm to ensure efficient bandwidth, however it harmed performance in certain cases, let us discuss this. Download the full RFC from here. https://www.rfc-editor.org/rfc/pdfrfc/rfc896.txt.pdf, I also included it in this lecture as a downloadable.
As described in RFC 1122, the delayed acknowledgment algorithm can delay ACKs by up to half a second which in combination with Nagle algorithm it can have devestating performance on communication
Download RFC 1122 here https://www.rfc-editor.org/rfc/pdfrfc/rfc1122.txt.pdf, I also included it as a downloadable on this lecture for more reading
Postgres Access Control https://www.postgresql.org/docs/current/auth-pg-hba-conf.html
In this video I learned how to decrypt TLS traffic with Wireshark so we can look at the traffic. Then took a look at how HTTP/2 streams are formed. The Magic HTTP/2 Stream is for sure interesting and the odd stream Ids are too!
HTTP/2 RFC
https://tools.ietf.org/html/rfc7540#section-3.5
https://http2.github.io/http2-spec/#SETTINGS_MAX_CONCURRENT_STREAMS
Connection Preface
HTTP/2 connections are opened with the preface '0x505249202a20485454502f322e300d0a0d0a534d0d0a0d0a', or "PRI * HTTP/2.0\r\n\r\nSM\r\n\r\n". The preface is designed to avoid the processing of frames by servers and intermediaries which support earlier versions of HTTP but not 2.0.
connection preface
Note: The client connection preface is selected so that a large
proportion of HTTP/1.1 or HTTP/1.0 servers and intermediaries do
not attempt to process further frames. Note that this does not
address the concerns raised in [TALKING].
export SSLKEYLOGFILE=/Users/HusseinNasser/tempkeys/key
:30 protocol ossifcations
13-30 http smuggling
From RFC (we discovered this in the video)
5.1.1 Stream Identifiers
Streams are identified with an unsigned 31-bit integer. Streams initiated by a client MUST use odd-numbered stream identifiers; those initiated by the server MUST use even-numbered stream identifiers. A stream identifier of zero (0x0) is used for connection control messages; the stream identifier of zero cannot be used to establish a new stream.
In this lecture I wireshark MongoDB and describe how it works in the wire. We will be Decrypting TLS traffic, showing the mongo protocol, cursors and more
Source code for MongoDB source code here https://github.com/hnasr/javascript_playground/blob/master/mongodb-js/test.js
Source Code
https://github.com/hnasr/javascript_playground/tree/master/server-sent-events
In this lecture I explain how TLS 0-RTT work
OpenCourser helps millions of learners each year. People visit us to learn workspace skills, ace their exams, and nurture their curiosity.
Our extensive catalog contains over 50,000 courses and twice as many books. Browse by search, by topic, or even by career interests. We'll match you to the right resources quickly.
Find this site helpful? Tell a friend about us.
We're supported by our community of learners. When you purchase or subscribe to courses and programs or purchase books, we may earn a commission from our partners.
Your purchases help us maintain our catalog and keep our servers humming without ads.
Thank you for supporting OpenCourser.