We may earn an affiliate commission when you visit our partners.
Large Language Models
Foundation Models from the Ground Up
Matei Zaharia,
Sam Raymond,
Chengyin Eng,
and
Joseph Bradley
Matei Zaharia,
Sam Raymond,
Chengyin Eng,
and
Joseph Bradley
This course dives into the details of LLM foundation models. You will learn the innovations that led to the proliferation of transformer-based architectures, from encoder models (BERT), to decoder models (GPT), to encoder-decoder models (T5). You will also learn about the recent breakthroughs that led to applications like ChatGPT. You will gain understanding about the latest advances that continue to improve LLM functionality including Flash Attention, LoRa, AliBi, and PEFT methods. The course concludes with an overview of multi-modal LLM developments to address NLP problems involving a combination of text, audio, and visual components.
Register for this course and see more details by visiting:
OpenCourser.com/course/nyisnw/large
What's inside
Learning objectives
- Describe the components and theory behind foundation models, including the attention mechanism, encoder and decoder architectures.
- Articulate the developments in the evolution of gpt models that were critical in the creation of popular llms like chatgpt.
- Explain and implement the latest advances that improve llm functionality, including fast attention, alibi, and peft methods.
- Gain insights into multi-modal applications of generative ai (genai) / llms involving a combination of text, audio, and visual elements.
- Describe the components and theory behind foundation models, including the attention mechanism, encoder and decoder architectures.
- Articulate the developments in the evolution of gpt models that were critical in the creation of popular llms like chatgpt.
- Explain and implement the latest advances that improve llm functionality, including fast attention, alibi, and peft methods.
- Gain insights into multi-modal applications of generative ai (genai) / llms involving a combination of text, audio, and visual elements.
Syllabus
Module 1 - Transformer Architecture: Attention & Transformer Fundamentals
Module 2 - Efficient Fine Tuning
Module 3 - Deployment and Hardware Considerations
Read more
Syllabus
Module 1 - Transformer Architecture: Attention & Transformer Fundamentals
Module 2 - Efficient Fine Tuning
Module 3 - Deployment and Hardware Considerations
Read more
Module 4 - Beyond Text-Based LLMs: Multi-Modality
Good to know
Good to know
Know what's good ,
what to watch for , and
possible dealbreakers
Examines transformer architectures, which are foundational to Generative AI (GenAI) / LLMs and used widely in NLP
Taught by Matei Zaharia, Sam Raymond, Chengyin Eng, and Joseph Bradley, who are recognized for their work in foundational models and LLM theory and practice
Develops skills in transformer architectures and techniques that are core for working with Generative AI (GenAI) / LLMs
Covers the latest advances in LLM functionality, including Fast Attention, AliBi, and PEFT methods, which are highly sought after by industry
May require prerequisites and/or advisory prerequisites before enrolling
Save this course
Save Large Language Models: Foundation Models from the Ground Up to your list so you can find it easily later:
Save
Save
Activities
Coming soon
We're preparing activities for
Large Language Models: Foundation Models from the Ground Up.
These are activities you can do either before, during, or after a course.
Career center
Learners who complete Large Language Models: Foundation Models from the Ground Up will develop knowledge and skills
that may be useful to these careers:
Machine Learning Engineer
Machine Learning Engineer
Machine Learning Engineers develop, test, and deploy machine learning models. A core part of this job is understanding the theory behind machine learning, especially the fundamentals of neural networks and transformer models. This course will provide the necessary background knowledge on these important concepts, ensuring you are well-prepared to enter the field as a Machine Learning Engineer.
Data Scientist
Data Scientist
Data Scientists use machine learning to derive actionable insights from data. The models and algorithms they design are often transformer-based, so a strong theoretical understanding of how these models work is required. This course gives a detailed overview of the inner workings of transformer models, ranging from their architecture to their fine-tuning and hardware considerations. Taking this course will help you build a strong foundation for a successful career as a Data Scientist.
Natural Language Processing Engineer
Natural Language Processing Engineer
Natural Language Processing Engineers work on developing and deploying AI-powered applications that process and understand human language. Since transformer models are dominant in the field of NLP, a deep understanding of their architecture and functionality is crucial. This course offers a comprehensive understanding of the fundamentals and latest advancements in transformer-based NLP models. Completing this course will give you a significant competitive edge for roles in Natural Language Processing.
AI Researcher
AI Researcher
AI Researchers develop new machine learning algorithms and techniques to solve complex problems. Understanding the theoretical underpinnings of transformer models is essential for developing new models and improving existing ones. This course covers advanced topics such as Flash Attention, LoRa, AliBi, and PEFT methods. Completing this course will provide you with a cutting-edge understanding of transformer models that will serve you well in a career as an AI Researcher.
Software Engineer
Software Engineer
Software Engineers who specialize in developing AI-powered applications need to have a strong understanding of machine learning models, including transformer models. This course provides an accessible overview of transformer models and their applications, with a focus on efficient fine-tuning and hardware considerations. Completing this course will give you a solid foundation to develop and deploy robust AI-powered software systems.
Data Analyst
Data Analyst
Data Analysts use data to solve business problems. While not strictly required, a strong grounding in transformer models can give you an edge in this field. This course introduces transformer models and their applications from the ground up, giving you the skills to identify opportunities for applying transformer-based solutions in a data analysis context.
Product Manager
Product Manager
Product Managers are responsible for developing and managing the roadmap for a product. Having a basic understanding of transformer models and their applications can be beneficial in this role, especially when it comes to developing AI-powered features. This course gives you a broad overview of transformer models, allowing you to engage in informed discussions with your team and make better decisions about product development.
Business Analyst
Business Analyst
Business Analysts work with stakeholders to identify and solve business problems. As many companies leverage AI-powered solutions, Business Analysts with a basic understanding of transformer models can gain an edge. This course provides a foundational understanding of transformer models and their applications, enabling you to effectively communicate with technical teams and contribute to data-driven problem-solving initiatives.
UX Designer
UX Designer
UX Designers create user experiences for digital products. While not strictly required, having a basic understanding of transformer models can be beneficial in this field, especially when it comes to designing AI-powered user interfaces. This course introduces transformer models and their applications, giving you the background knowledge to understand the capabilities and limitations of AI in the context of user experience design.
Technical Writer
Technical Writer
Technical Writers create documentation for technical products. Having a basic understanding of transformer models can be beneficial in this field, especially when it comes to writing documentation for AI-powered products. This course introduces transformer models and their applications, giving you the background knowledge to effectively communicate complex technical concepts to a non-technical audience.
Science Writer
Science Writer
Science Writers communicate complex scientific concepts to a general audience. A basic understanding of transformer models can be beneficial in this field, especially when it comes to writing about AI and machine learning. This course introduces transformer models and their applications, giving you the background knowledge to accurately and engagingly convey these concepts to your readers.
Marketing Manager
Marketing Manager
Marketing Managers develop and execute marketing campaigns. Understanding transformer models and their applications can be useful in leveraging AI for marketing purposes. This course provides a broad understanding of transformer models, enabling you to engage with marketing teams and make informed decisions about AI-powered marketing strategies.
Sales Manager
Sales Manager
Sales Managers lead sales teams and develop sales strategies. While not strictly required, having a basic understanding of transformer models can be beneficial in understanding AI-driven sales tools and techniques. This course provides an accessible overview of transformer models and their applications, giving you the opportunity to build on your knowledge of AI in the context of sales and business development.
Operations Manager
Operations Manager
Operations Managers oversee the day-to-day operations of an organization. While not a direct requirement, a basic understanding of transformer models can be helpful in understanding AI-powered automation and optimization tools. This course introduces transformer models and their applications, giving you the background knowledge to make informed decisions about adopting AI solutions in your operations.
Project Manager
Project Manager
Project Managers plan and execute projects. Having a basic understanding of transformer models can be helpful in managing AI-powered projects. This course provides an accessible overview of transformer models and their applications, giving you the knowledge to effectively communicate with technical teams and make informed decisions about AI-related project deliverables.
For more career information including salaries, visit:
OpenCourser.com/course/nyisnw/large
Reading list
We've selected six books
that we think will supplement your
learning. Use these to
develop background knowledge, enrich your coursework, and gain a
deeper understanding of the topics covered in
Large Language Models: Foundation Models from the Ground Up.
This blog provides updates on the latest research and developments in AI from Google, including LLMs and transformer neural networks.
Provides a comprehensive overview of statistical natural language processing. It would be a valuable reference for students who want to understand the mathematical foundations of NLP.
Provides a comprehensive overview of natural language processing. It covers many of the techniques and algorithms that are used in foundational models like LLMs.
Provides a comprehensive overview of speech and language processing. It would be useful as background reading for students who are new to NLP.
Provides a comprehensive overview of statistical learning, including a chapter on natural language processing. It would be helpful as background reading for students who are new to machine learning.
Provides a practical introduction to deep learning, including a chapter on natural language processing. It would be helpful as background reading for students who are new to deep learning.
For more information about how these books relate to this course, visit:
OpenCourser.com/course/nyisnw/large
Share
Similar courses
Here are nine courses similar to
Large Language Models: Foundation Models from the Ground Up.
LLMs Mastery: Complete Guide to Transformers & Generative AI
OpenCourser.com/course/mma4rz/llms
LLMs Mastery: Complete Guide to Transformers & Generative...
Most relevant
Natural Language Processing with Attention Models
OpenCourser.com/course/lo38w9/natural
Natural Language Processing with Attention Models
Most relevant
Create Image Captioning Models
OpenCourser.com/course/zn5xo4/create
Create Image Captioning Models
Most relevant
Encoder-Decoder Architecture
OpenCourser.com/course/iho664/encoder
Encoder-Decoder Architecture
Most relevant
Encoder-Decoder Architecture with Google Cloud
OpenCourser.com/course/degyrm/encoder
Encoder-Decoder Architecture with Google Cloud
Most relevant
Learn Everything about Full-Stack Generative AI, LLM MODELS
OpenCourser.com/course/xacvmg/learn
Learn Everything about Full-Stack Generative AI, LLM...
Most relevant
Efficiently Serving LLMs
OpenCourser.com/course/0umlmh/efficiently
Efficiently Serving LLMs
Most relevant
Create Image Captioning Models with Google Cloud
OpenCourser.com/course/nnskhp/create
Create Image Captioning Models with Google Cloud
Most relevant
Generative AI Language Modeling with Transformers
OpenCourser.com/course/kn2svd/generative
Generative AI Language Modeling with Transformers
Most relevant
Effort
4 to 8 hours per week for 4 weeks
Level
Advanced
Via
edX
Institution
Databricks
Instructors
Matei Zaharia
Sam Raymond
Chengyin Eng
Joseph Bradley
Language
English
Transcript
English
Good to know
Examines transformer architectures, which are foundational to Generative AI (GenAI) / LLMs and used widely in NLP
Taught by Matei Zaharia, Sam Raymond, Chengyin Eng, and Joseph Bradley, who are recognized for their work in foundational models and LLM theory and practice
Develops skills in transformer architectures and techniques that are core for working with Generative AI (GenAI) / LLMs
Covers the latest advances in LLM functionality, including Fast Attention, AliBi, and PEFT methods, which are highly sought after by industry
May require prerequisites and/or advisory prerequisites before enrolling
Our mission
OpenCourser helps millions of learners each year. People visit us to learn workspace skills, ace their exams, and nurture their curiosity.
Our extensive catalog contains over 50,000 courses and twice as many books. Browse by search, by topic, or even by career interests. We'll match you to the right resources quickly.
Find this site helpful? Tell a friend about us.
Affiliate disclosure
We're supported by our community of learners. When you purchase or subscribe to courses and programs or purchase books, we may earn a commission from our partners.
Your purchases help us maintain our catalog and keep our servers humming without ads.
Thank you for supporting OpenCourser.
© 2016 - 2024 OpenCourser