Large language models (LLMs) are trained on human-generated text, but additional methods are needed to align an LLM with human values and preferences.
Large language models (LLMs) are trained on human-generated text, but additional methods are needed to align an LLM with human values and preferences.
Reinforcement Learning from Human Feedback (RLHF) is currently the main method for aligning LLMs with human values and preferences. RLHF is also used for further tuning a base LLM to align with values and preferences that are specific to your use case.
In this course, you will gain a conceptual understanding of the RLHF training process, and then practice applying RLHF to tune an LLM. You will:
1. Explore the two datasets that are used in RLHF training: the “preference” and “prompt” datasets.
2. Use the open source Google Cloud Pipeline Components Library, to fine-tune the Llama 2 model with RLHF.
3. Assess the tuned LLM against the original base model by comparing loss curves and using the “Side-by-Side (SxS)” method.
OpenCourser helps millions of learners each year. People visit us to learn workspace skills, ace their exams, and nurture their curiosity.
Our extensive catalog contains over 50,000 courses and twice as many books. Browse by search, by topic, or even by career interests. We'll match you to the right resources quickly.
Find this site helpful? Tell a friend about us.
We're supported by our community of learners. When you purchase or subscribe to courses and programs or purchase books, we may earn a commission from our partners.
Your purchases help us maintain our catalog and keep our servers humming without ads.
Thank you for supporting OpenCourser.