We may earn an affiliate commission when you visit our partners.
Matei Zaharia,
Sam Raymond,
Chengyin Eng,
and
Joseph Bradley
This course dives into the details of LLM foundation models. You will learn the innovations that led to the proliferation of transformer-based architectures, from encoder models (BERT), to decoder models (GPT), to encoder-decoder models (T5). You will also learn about the recent breakthroughs that led to applications like ChatGPT. You will gain understanding about the latest advances that continue to improve LLM functionality including Flash Attention, LoRa, AliBi, and PEFT methods. The course concludes with an overview of multi-modal LLM developments to address NLP problems involving a combination of text, audio, and visual components.
Register for this course and see more details by visiting:
OpenCourser.com/course/nyisnw/large
Three deals to help you save
We found three deals and offers that may be relevant to this course.
Save money when you learn.
All coupon codes, vouchers, and discounts are applied automatically unless otherwise noted.
Use code at checkout.
Valid until December 4
Save on Select Programs
Stay ahead in a competitive job market and save with these programs.
Take
30%
off
EDXCYBER24
Valid for a limited time only
Future-proof your career
Access O'Reilly books, live events, courses, and more. Save with an annual subscription.
Take
15%
off
What's inside
Learning objectives
- Describe the components and theory behind foundation models, including the attention mechanism, encoder and decoder architectures.
- Articulate the developments in the evolution of gpt models that were critical in the creation of popular llms like chatgpt.
- Explain and implement the latest advances that improve llm functionality, including fast attention, alibi, and peft methods.
- Gain insights into multi-modal applications of generative ai (genai) / llms involving a combination of text, audio, and visual elements.
- Describe the components and theory behind foundation models, including the attention mechanism, encoder and decoder architectures.
- Articulate the developments in the evolution of gpt models that were critical in the creation of popular llms like chatgpt.
- Explain and implement the latest advances that improve llm functionality, including fast attention, alibi, and peft methods.
- Gain insights into multi-modal applications of generative ai (genai) / llms involving a combination of text, audio, and visual elements.
Syllabus
Module 1 - Transformer Architecture: Attention & Transformer Fundamentals
Module 2 - Efficient Fine Tuning
Module 3 - Deployment and Hardware Considerations
Read more
Syllabus
Module 1 - Transformer Architecture: Attention & Transformer Fundamentals
Module 2 - Efficient Fine Tuning
Module 3 - Deployment and Hardware Considerations
Read more
Module 4 - Beyond Text-Based LLMs: Multi-Modality
Good to know
Good to know
Know what's good ,
what to watch for , and
possible dealbreakers
Examines transformer architectures, which are foundational to Generative AI (GenAI) / LLMs and used widely in NLP
Taught by Matei Zaharia, Sam Raymond, Chengyin Eng, and Joseph Bradley, who are recognized for their work in foundational models and LLM theory and practice
Develops skills in transformer architectures and techniques that are core for working with Generative AI (GenAI) / LLMs
Covers the latest advances in LLM functionality, including Fast Attention, AliBi, and PEFT methods, which are highly sought after by industry
May require prerequisites and/or advisory prerequisites before enrolling
Save this course
Save Large Language Models: Foundation Models from the Ground Up to your list so you can find it easily later:
Save
Save
Activities
Be better prepared
before
your course. Deepen your understanding
during
and
after
it. Supplement your coursework and achieve mastery of the topics covered
in Large Language Models: Foundation Models from the Ground Up with these
activities:
Become a peer mentor in the course forum
Show steps
Contribute to the success of your peers by sharing your knowledge and providing guidance through the course forum.
Browse courses on
Mentoring
Show steps
-
Offer support and assistance to other students
-
Answer questions and provide clarifications
-
Share your own experiences and insights
Read "Deep Learning" by Ian Goodfellow, Yoshua Bengio, and Aaron Courville
Show steps
Build a solid foundation by reviewing this seminal textbook, gaining insights into deep learning concepts before the course begins.
View
Deep Learning
on Amazon
Show steps
-
Read chapters 1-4 and summarize key concepts
-
Complete practice exercises at the end of each chapter
-
Discuss key concepts with peers or mentors
Complete Stanford's Convolutional Neural Networks Tutorial
Show steps
Strengthen your foundational understanding by completing this guided tutorial, providing practical experience with CNN architectures.
Browse courses on
Convolutional Neural Networks
Show steps
-
Follow the video and text walkthroughs
-
Implement the code examples provided
-
Apply the techniques to your own dataset
Five other activities
Expand to see all activities and additional details
Show all eight activities
Solve LeetCode problems on transformer architectures and attention mechanisms
Show steps
Enhance your problem-solving abilities and deepen your understanding of transformer architectures and attention mechanisms through practice.
Browse courses on
Transformer Architectures
Show steps
-
Select problems on transformer architectures and attention mechanisms
-
Solve the problems independently
-
Review your solutions and learn from mistakes
Attend a conference on transformer architectures and deep learning
Show steps
Connect with experts and learn about the latest advancements in transformer architectures and deep learning.
Browse courses on
Networking
Show steps
-
Research and identify relevant conferences
-
Register and attend the conference
-
Network with speakers, attendees, and industry professionals
Build a blog post or article on the history of transformer architectures
Show steps
Solidify your understanding by researching and writing about the evolution and key advancements in transformer architectures.
Browse courses on
Transformer Architectures
Show steps
-
Research the history of transformer architectures
-
Organize your findings into an outline
-
Write and edit your blog post or article
-
Publish and share your work
Volunteer as a research assistant on a project related to transformer architectures
Show steps
Gain invaluable practical experience and contribute to the advancement of transformer architectures through research.
Browse courses on
Transformer Architectures
Show steps
-
Identify and contact potential research opportunities
-
Assist with data collection, analysis, or model development
-
Present your findings at conferences or publish in journals
Implement a transformer-based language model from scratch
Show steps
Challenge yourself by building a transformer-based language model from scratch, gaining hands-on experience and deepening your understanding.
Browse courses on
Transformer Architectures
Show steps
-
Design the architecture of the model
-
Implement the model in a programming language
-
Train the model on a text dataset
-
Evaluate the performance of the model
Become a peer mentor in the course forum
Show steps
Contribute to the success of your peers by sharing your knowledge and providing guidance through the course forum.
Browse courses on
Mentoring
Show steps
Show steps
Show steps
- Offer support and assistance to other students
- Answer questions and provide clarifications
- Share your own experiences and insights
Read "Deep Learning" by Ian Goodfellow, Yoshua Bengio, and Aaron Courville
Show steps
Build a solid foundation by reviewing this seminal textbook, gaining insights into deep learning concepts before the course begins.
View
Deep Learning
on Amazon
Show steps
Show steps
Show steps
- Read chapters 1-4 and summarize key concepts
- Complete practice exercises at the end of each chapter
- Discuss key concepts with peers or mentors
Complete Stanford's Convolutional Neural Networks Tutorial
Show steps
Strengthen your foundational understanding by completing this guided tutorial, providing practical experience with CNN architectures.
Browse courses on
Convolutional Neural Networks
Show steps
Show steps
Show steps
- Follow the video and text walkthroughs
- Implement the code examples provided
- Apply the techniques to your own dataset
Five other activities
Expand to see all activities and additional details
Show all eight activities
Solve LeetCode problems on transformer architectures and attention mechanisms
Show steps
Enhance your problem-solving abilities and deepen your understanding of transformer architectures and attention mechanisms through practice.
Browse courses on
Transformer Architectures
Show steps
Show steps
Show steps
- Select problems on transformer architectures and attention mechanisms
- Solve the problems independently
- Review your solutions and learn from mistakes
Attend a conference on transformer architectures and deep learning
Show steps
Connect with experts and learn about the latest advancements in transformer architectures and deep learning.
Browse courses on
Networking
Show steps
Show steps
Show steps
- Research and identify relevant conferences
- Register and attend the conference
- Network with speakers, attendees, and industry professionals
Build a blog post or article on the history of transformer architectures
Show steps
Solidify your understanding by researching and writing about the evolution and key advancements in transformer architectures.
Browse courses on
Transformer Architectures
Show steps
Show steps
Show steps
- Research the history of transformer architectures
- Organize your findings into an outline
- Write and edit your blog post or article
- Publish and share your work
Volunteer as a research assistant on a project related to transformer architectures
Show steps
Gain invaluable practical experience and contribute to the advancement of transformer architectures through research.
Browse courses on
Transformer Architectures
Show steps
Show steps
Show steps
- Identify and contact potential research opportunities
- Assist with data collection, analysis, or model development
- Present your findings at conferences or publish in journals
Implement a transformer-based language model from scratch
Show steps
Challenge yourself by building a transformer-based language model from scratch, gaining hands-on experience and deepening your understanding.
Browse courses on
Transformer Architectures
Show steps
Show steps
Show steps
- Design the architecture of the model
- Implement the model in a programming language
- Train the model on a text dataset
- Evaluate the performance of the model
Career center
Learners who complete Large Language Models: Foundation Models from the Ground Up will develop knowledge and skills
that may be useful to these careers:
Machine Learning Engineer
Machine Learning Engineer
Machine Learning Engineers develop, test, and deploy machine learning models. A core part of this job is understanding the theory behind machine learning, especially the fundamentals of neural networks and transformer models. This course will provide the necessary background knowledge on these important concepts, ensuring you are well-prepared to enter the field as a Machine Learning Engineer.
Data Scientist
Data Scientist
Data Scientists use machine learning to derive actionable insights from data. The models and algorithms they design are often transformer-based, so a strong theoretical understanding of how these models work is required. This course gives a detailed overview of the inner workings of transformer models, ranging from their architecture to their fine-tuning and hardware considerations. Taking this course will help you build a strong foundation for a successful career as a Data Scientist.
Natural Language Processing Engineer
Natural Language Processing Engineer
Natural Language Processing Engineers work on developing and deploying AI-powered applications that process and understand human language. Since transformer models are dominant in the field of NLP, a deep understanding of their architecture and functionality is crucial. This course offers a comprehensive understanding of the fundamentals and latest advancements in transformer-based NLP models. Completing this course will give you a significant competitive edge for roles in Natural Language Processing.
AI Researcher
AI Researcher
AI Researchers develop new machine learning algorithms and techniques to solve complex problems. Understanding the theoretical underpinnings of transformer models is essential for developing new models and improving existing ones. This course covers advanced topics such as Flash Attention, LoRa, AliBi, and PEFT methods. Completing this course will provide you with a cutting-edge understanding of transformer models that will serve you well in a career as an AI Researcher.
Software Engineer
Software Engineer
Software Engineers who specialize in developing AI-powered applications need to have a strong understanding of machine learning models, including transformer models. This course provides an accessible overview of transformer models and their applications, with a focus on efficient fine-tuning and hardware considerations. Completing this course will give you a solid foundation to develop and deploy robust AI-powered software systems.
Data Analyst
Data Analyst
Data Analysts use data to solve business problems. While not strictly required, a strong grounding in transformer models can give you an edge in this field. This course introduces transformer models and their applications from the ground up, giving you the skills to identify opportunities for applying transformer-based solutions in a data analysis context.
Product Manager
Product Manager
Product Managers are responsible for developing and managing the roadmap for a product. Having a basic understanding of transformer models and their applications can be beneficial in this role, especially when it comes to developing AI-powered features. This course gives you a broad overview of transformer models, allowing you to engage in informed discussions with your team and make better decisions about product development.
Business Analyst
Business Analyst
Business Analysts work with stakeholders to identify and solve business problems. As many companies leverage AI-powered solutions, Business Analysts with a basic understanding of transformer models can gain an edge. This course provides a foundational understanding of transformer models and their applications, enabling you to effectively communicate with technical teams and contribute to data-driven problem-solving initiatives.
UX Designer
UX Designer
UX Designers create user experiences for digital products. While not strictly required, having a basic understanding of transformer models can be beneficial in this field, especially when it comes to designing AI-powered user interfaces. This course introduces transformer models and their applications, giving you the background knowledge to understand the capabilities and limitations of AI in the context of user experience design.
Technical Writer
Technical Writer
Technical Writers create documentation for technical products. Having a basic understanding of transformer models can be beneficial in this field, especially when it comes to writing documentation for AI-powered products. This course introduces transformer models and their applications, giving you the background knowledge to effectively communicate complex technical concepts to a non-technical audience.
Science Writer
Science Writer
Science Writers communicate complex scientific concepts to a general audience. A basic understanding of transformer models can be beneficial in this field, especially when it comes to writing about AI and machine learning. This course introduces transformer models and their applications, giving you the background knowledge to accurately and engagingly convey these concepts to your readers.
Marketing Manager
Marketing Manager
Marketing Managers develop and execute marketing campaigns. Understanding transformer models and their applications can be useful in leveraging AI for marketing purposes. This course provides a broad understanding of transformer models, enabling you to engage with marketing teams and make informed decisions about AI-powered marketing strategies.
Sales Manager
Sales Manager
Sales Managers lead sales teams and develop sales strategies. While not strictly required, having a basic understanding of transformer models can be beneficial in understanding AI-driven sales tools and techniques. This course provides an accessible overview of transformer models and their applications, giving you the opportunity to build on your knowledge of AI in the context of sales and business development.
Operations Manager
Operations Manager
Operations Managers oversee the day-to-day operations of an organization. While not a direct requirement, a basic understanding of transformer models can be helpful in understanding AI-powered automation and optimization tools. This course introduces transformer models and their applications, giving you the background knowledge to make informed decisions about adopting AI solutions in your operations.
Project Manager
Project Manager
Project Managers plan and execute projects. Having a basic understanding of transformer models can be helpful in managing AI-powered projects. This course provides an accessible overview of transformer models and their applications, giving you the knowledge to effectively communicate with technical teams and make informed decisions about AI-related project deliverables.
For more career information including salaries, visit:
OpenCourser.com/course/nyisnw/large
Reading list
We've selected six books
that we think will supplement your
learning. Use these to
develop background knowledge, enrich your coursework, and gain a
deeper understanding of the topics covered in
Large Language Models: Foundation Models from the Ground Up.
This blog provides updates on the latest research and developments in AI from Google, including LLMs and transformer neural networks.
Provides a comprehensive overview of statistical natural language processing. It would be a valuable reference for students who want to understand the mathematical foundations of NLP.
Provides a comprehensive overview of natural language processing. It covers many of the techniques and algorithms that are used in foundational models like LLMs.
Provides a comprehensive overview of speech and language processing. It would be useful as background reading for students who are new to NLP.
Provides a comprehensive overview of statistical learning, including a chapter on natural language processing. It would be helpful as background reading for students who are new to machine learning.
Provides a practical introduction to deep learning, including a chapter on natural language processing. It would be helpful as background reading for students who are new to deep learning.
For more information about how these books relate to this course, visit:
OpenCourser.com/course/nyisnw/large
Share
Similar courses
Here are nine courses similar to
Large Language Models: Foundation Models from the Ground Up.
LLMs Mastery: Complete Guide to Transformers & Generative AI
OpenCourser.com/course/mma4rz/llms
LLMs Mastery: Complete Guide to Transformers & Generative...
Most relevant
Natural Language Processing with Attention Models
OpenCourser.com/course/lo38w9/natural
Natural Language Processing with Attention Models
Most relevant
Create Image Captioning Models
OpenCourser.com/course/zn5xo4/create
Create Image Captioning Models
Most relevant
Encoder-Decoder Architecture
OpenCourser.com/course/iho664/encoder
Encoder-Decoder Architecture
Most relevant
Encoder-Decoder Architecture with Google Cloud
OpenCourser.com/course/degyrm/encoder
Encoder-Decoder Architecture with Google Cloud
Most relevant
Learn Everything about Full-Stack Generative AI, LLM MODELS
OpenCourser.com/course/xacvmg/learn
Learn Everything about Full-Stack Generative AI, LLM...
Most relevant
Efficiently Serving LLMs
OpenCourser.com/course/0umlmh/efficiently
Efficiently Serving LLMs
Most relevant
Create Image Captioning Models with Google Cloud
OpenCourser.com/course/nnskhp/create
Create Image Captioning Models with Google Cloud
Most relevant
Generative AI Language Modeling with Transformers
OpenCourser.com/course/kn2svd/generative
Generative AI Language Modeling with Transformers
Most relevant
Effort
4 to 8 hours per week for 4 weeks
Level
Advanced
Via
edX
Institution
Databricks
Instructors
Matei Zaharia
Sam Raymond
Chengyin Eng
Joseph Bradley
Language
English
Transcript
English
Good to know
Examines transformer architectures, which are foundational to Generative AI (GenAI) / LLMs and used widely in NLP
Taught by Matei Zaharia, Sam Raymond, Chengyin Eng, and Joseph Bradley, who are recognized for their work in foundational models and LLM theory and practice
Develops skills in transformer architectures and techniques that are core for working with Generative AI (GenAI) / LLMs
Covers the latest advances in LLM functionality, including Fast Attention, AliBi, and PEFT methods, which are highly sought after by industry
May require prerequisites and/or advisory prerequisites before enrolling
Our mission
OpenCourser helps millions of learners each year. People visit us to learn workspace skills, ace their exams, and nurture their curiosity.
Our extensive catalog contains over 50,000 courses and twice as many books. Browse by search, by topic, or even by career interests. We'll match you to the right resources quickly.
Find this site helpful? Tell a friend about us.
Affiliate disclosure
We're supported by our community of learners. When you purchase or subscribe to courses and programs or purchase books, we may earn a commission from our partners.
Your purchases help us maintain our catalog and keep our servers humming without ads.
Thank you for supporting OpenCourser.
© 2016 - 2024 OpenCourser