OpenCourser
Learn skills that shape your future
Visit OpenCourser.com
Deep Understanding of Large Language Models (LLMs): Architecture, Training, and Mechanisms
Description
Large Language Models (LLMs) like ChatGPT, GPT- But most courses only teach you how to use LLMs. This 90+ hour intensive course teaches you how they actually work — and how to dissect them using machine-learning and mechanistic interpretability methods.
This is a deep, end-to-end exploration of transformer architectures, self-attention mechanisms, embeddings layers, training pipelines, and inference strategies — with hands-on Python and PyTorch code at every step.
Deep Understanding of Large Language Models (LLMs): Architecture, Training, and Mechanisms
Description
Large Language Models (LLMs) like ChatGPT, GPT- But most courses only teach you how to use LLMs. This 90+ hour intensive course teaches you how they actually work — and how to dissect them using machine-learning and mechanistic interpretability methods.
This is a deep, end-to-end exploration of transformer architectures, self-attention mechanisms, embeddings layers, training pipelines, and inference strategies — with hands-on Python and PyTorch code at every step.
Whether your goal is to build your own transformer from scratch, fine-tune existing models, or understand the mathematics and engineering behind state-of-the-art generative AI, this course will give you the foundation and tools you need.
What You’ll Learn
The complete architecture of LLMs — tokenization, embeddings, encoders, decoders, attention heads, feedforward networks, and layer normalization
Mathematics of attention mechanisms — dot-product attention, multi-head attention, positional encoding, causal masking, probabilistic token selection
Training LLMs — optimization (Adam, AdamW), loss functions, gradient accumulation, batch processing, learning-rate schedulers, regularization (L1, L2, decorrelation), gradient clipping
Fine-tuning and prompt engineering for downstream NLP tasks, system-tuning
Evaluation metrics — perplexity, accuracy, and benchmark datasets such as MAUVE, HellaSwag, SuperGLUE, and ways to assess bias and fairness
Practical PyTorch implementations of transformers, attention layers, and language model training loops, custom classes, custom loss functions
Inference techniques — greedy decoding, beam search, top-k sampling, temperature scaling
Scaling laws and trade-offs between model size, training data, and performance
Limitations and biases in LLMs — interpretability, ethical considerations, and responsible AI
Decoder-only transformers
Embeddings, including token embeddings and positional embeddings
Sampling techniques — methods for generating new text, including top-p, top-k, multinomial, and greedy
Why This Course Is Different
93+ hours of HD video lectures — blending theory, code, and practical application
Code challenges in every section — with full, downloadable solutions
Builds from first principles — starting from basic Python/Numpy implementations and progressing to full PyTorch LLMs
Suitable for researchers, engineers, and advanced learners who want to go beyond “black box” API usage
Clear explanations without dumbing down the content — intensive but approachable
Who Is This Course For?
Machine learning engineers and data scientists
AI researchers and NLP specialists
Software developers interested in deep learning and generative AI
Graduate students or self-learners with intermediate Python skills and basic ML knowledge
Technologies & Tools Covered
Python and PyTorch for deep learning
NumPy and Matplotlib for numerical computing and visualization
Google Colab for free GPU access
Hugging Face Transformers for working with pre-trained models
Tokenizers and text preprocessing tools
Implement Transformers in PyTorch, fine-tune LLMs, decode with attention mechanisms, and probe model internals
What if you have questions about the material?
This course has a Q&A (question and answer) section where you can post your questions about the course material (about the maths, statistics, coding, or machine learning aspects). I try to answer all questions within a day. You can also see all other questions and answers, which really improves how much you can learn. And you can contribute to the Q&A by posting to ongoing discussions.
By the end of this course, you won’t just know how to work with LLMs — you’ll understand why they work the way they do, and be able to design, train, evaluate, and deploy your own transformer-based language models.
Enroll now and start mastering Large Language Models from the ground up.
Save
Save
Save
Save
Save
Save
Save
Save
Save
Verified partner policies & purchase protections
Free preview video lectures available for all courses; optional 7-day free trial on Personal Plan subscriptions (cancel before trial ends to avoid conversion).
30-day money-back guarantee on all eligible individual course purchases (provided course content has not been substantially consumed or downloaded). Subscription fees are non-refundable once billed.
Lifetime access to purchased individual courses and future updates, verifiable certificate of completion, and direct student Q&A forums.
OpenCourser helps millions of learners each year. People visit us to learn workspace skills, ace their exams, and nurture their curiosity.
Our extensive catalog contains over 50,000 courses and twice as many books. Browse by search, by topic, or even by career interests. We'll match you to the right resources quickly.
Find this site helpful? Tell a friend about us.
We're supported by our community of learners. When you purchase or subscribe to courses and programs or purchase books, we may earn a commission from our partners.
Your purchases help us maintain our catalog and keep our servers humming without ads.
Thank you for supporting OpenCourser.
