Fundamentals of Reinforcement Learning from Coursera

Reinforcement Learning is a subfield of Machine Learning, but is also a general purpose formalism for automated decision-making and AI. This course introduces you to statistical learning techniques where an agent explicitly takes actions and interacts with the world. Understanding the importance and challenges of learning agents that make decisions is of vital importance today, with more and more companies interested in interactive agents and intelligent decision-making.

This course introduces you to the fundamentals of Reinforcement Learning. When you finish this course, you will:

- Formalize problems as Markov Decision Processes

- Understand basic exploration methods and the exploration/exploitation tradeoff

- Understand value functions, as a general-purpose tool for optimal decision-making

- Know how to implement dynamic programming as an efficient solution approach to an industrial control problem

This course teaches you the key concepts of Reinforcement Learning, underlying classic and modern algorithms in RL. After completing this course, you will be able to start using RL for real problems, where you have or can specify the MDP.

This is the first course of the Reinforcement Learning Specialization.

What's inside

Syllabus

Welcome to the Course!

Welcome to: Fundamentals of Reinforcement Learning, the first course in a four-part specialization on Reinforcement Learning brought to you by the University of Alberta, Onlea, and Coursera. In this pre-course module, you'll be introduced to your instructors, get a flavour of what the course has in store for you, and be given an in-depth roadmap to help make your journey through this specialization as smooth as possible.

An Introduction to Sequential Decision-Making

For the first week of this course, you will learn how to understand the exploration-exploitation trade-off in sequential decision-making, implement incremental algorithms for estimating action-values, and compare the strengths and weaknesses to different algorithms for exploration. For this week’s graded assessment, you will implement and test an epsilon-greedy agent.

Markov Decision Processes

When you’re presented with a problem in industry, the first and most important step is to translate that problem into a Markov Decision Process (MDP). The quality of your solution depends heavily on how well you do this translation. This week, you will learn the definition of MDPs, you will understand goal-directed behavior and how this can be obtained from maximizing scalar rewards, and you will also understand the difference between episodic and continuing tasks. For this week’s graded assessment, you will create three example tasks of your own that fit into the MDP framework.

Value Functions & Bellman Equations

Once the problem is formulated as an MDP, finding the optimal policy is more efficient when using value functions. This week, you will learn the definition of policies and value functions, as well as Bellman equations, which is the key technology that all of our algorithms will use.

Dynamic Programming

This week, you will learn how to compute value functions and optimal policies, assuming you have the MDP model. You will implement dynamic programming to compute value functions and optimal policies and understand the utility of dynamic programming for industrial applications and problems. Further, you will learn about Generalized Policy Iteration as a common template for constructing algorithms that maximize reward. For this week’s graded assessment, you will implement an efficient dynamic programming agent in a simulated industrial control problem.

Good to know

Know what's good

, what to watch for

, and possible dealbreakers

Suitable for learners completely new to Reinforcement Learning

Intermediate learners strengthen foundational understanding of RL

Teaches importance and challenges of using learning agents for decision-making

Suitable for learners working with MDPs, or those who can specify MDPs

Suited for learners interested in real-world applications of RL

Instructors have strong reputations in field of RL

Reviews summary

Fundamentals of reinforcement learning

learners say this Coursera course is an excellent starting point for understanding and applying Reinforcement Learning (RL). It combines theory and practice, using readings, videos, quizzes, and programming assignments to reinforce learning. The course is based on the textbook "Reinforcement Learning: An Introduction" by Sutton and Barto and provides supplementary guest lectures by other experts in the field. It follows the first four chapters of the book and covers concepts such as MDPs, value iteration, policy iteration, and dynamic programming. learners appreciate the course's clear explanations, well-structured content, and helpful programming assignments. They find that the combination of video lectures, quizzes, and assignments effectively reinforces the material. The programming assignments are praised for providing hands-on practice and helping learners develop a deeper understanding of the underlying concepts. Some learners suggest that the course could be improved by providing more challenging coding assignments, more rigorous explanations of mathematical formulas, and a more comprehensive capstone project. They also recommend that learners read the textbook before watching the videos to get the most out of the course.

The course has a strong theoretical focus, with an emphasis on understanding the underlying mathematical concepts of RL. This can be both a strength and a weakness, depending on the learner's background and interests.

This course is largely positive, with learners expressing appreciation for the clear explanations, well-structured content, and helpful programming assignments.

While the course provides a solid theoretical foundation, it could benefit from more practical examples and applications of RL. This would help learners see how the concepts they are learning can be applied to real-world problems.

"It is ineffective in teaching the practicality, i.e, how real word problem can be related, what kind of problems can be solved by these methods."

"For those who want to pass the course, I recommend that they have knowledge of Python coding, if they are at an intermediate level, that is great."

The instructors are knowledgeable and passionate about the subject matter, but their delivery can be somewhat dry and monotonous. Some learners may prefer a more engaging and interactive teaching style.

"The instructors are friendly, which creates a pleasant learning atmosphere. However, there is room for improvement in the teaching of mathematical formulas. These are often covered too quickly and not explained sufficiently, making it very difficult to understand."

"I think the course's logic structure is so good. Even though I have read the textbook several times, I surprisingly found that many things I didn't understand or ignored when I was reading were clarified. The only thing I am not satisfied with is, the assignments (especially programming assignments) are too easy and the peer-graded assignment made me upset because I couldn't get my certificate immediately!"

"The verbal explanations during videos are very informative but not enough. Sometime a sentence that is said in 2 seconds require 10 minutes of work to understand."

"To be honest I didn't like videos in the course. Lectors read prepared text as robots. No pauses in places that are hard to understand. I had to do lots of replays to understand vids."

"Also, it would be great to have some more lectures that dig deeper into "alternate" representation of Bellman equations (we are thrown this question in the quiz, but some working professionals, like myself can be quite rusty in English<=>Notation mindset, but that's a "very" small nitpick item."

Activities

Be better prepared before your course. Deepen your understanding during and after it. Supplement your coursework and achieve mastery of the topics covered in Fundamentals of Reinforcement Learning with these activities:

Read "Reinforcement Learning: An Introduction, Second Edition" by Sutton and Barto

Show steps

This book is the seminal work on reinforcement learning and provides a comprehensive overview of the field. Reading this book will give students a strong theoretical foundation for the course.

View Reinforcement Learning, second edition: An... on Amazon

Show steps

Read Chapter 1: Introduction
Read Chapter 2: Background: Markov Decision Processes
Work through the exercises at the end of each chapter.

Review linear algebra concepts

Show steps

Linear algebra forms the mathematical foundation for Reinforcement Learning. Reviewing these concepts will help ensure a solid understanding for when they are applied later in the course.

Browse courses on Linear Algebra

Show steps

Review matrix operations, including addition, subtraction, multiplication, and determinants.
Practice solving systems of linear equations using Gaussian elimination.
Identify and classify different types of matrices, such as symmetric, orthogonal, and invertible matrices.

Mentor other students in the course

Show steps

Mentoring other students in the course is a great way to reinforce your own learning and help others to succeed.

Show steps

Join the course discussion forum.
Answer questions from other students.
Provide feedback on other students' work.

Four other activities

Expand to see all activities and additional details

Show all seven activities

Solve reinforcement learning problems using dynamic programming

Show steps

Dynamic programming is a powerful technique for solving reinforcement learning problems. Completing these practice drills will help students develop their problem-solving skills using this technique.

Browse courses on Dynamic programming

Show steps

Implement the value iteration algorithm to solve a simple gridworld problem.
Implement the policy iteration algorithm to solve a simple gridworld problem.
Compare the performance of the two algorithms on a variety of gridworld problems.

Follow the Reinforcement Learning Specialization on Coursera

Show steps

The Reinforcement Learning Specialization on Coursera provides a comprehensive overview of the field. Completing this specialization will give students a strong foundation for the course and beyond.

Browse courses on Coursera

Show steps

Complete the first course in the specialization: Fundamentals of Reinforcement Learning
Complete the second course in the specialization: Advanced Reinforcement Learning
Complete the third course in the specialization: Deep Reinforcement Learning

Build a reinforcement learning agent to play a game

Show steps

Building a reinforcement learning agent to play a game is a challenging but rewarding project. Completing this project will give students the opportunity to apply the concepts they learn in the course to a real-world problem.

Show steps

Choose a simple game to implement, such as tic-tac-toe or Connect Four.
Design the state space and action space for the game.
Implement a reinforcement learning algorithm, such as Q-learning or SARSA, to train the agent.
Test the performance of the agent against a human opponent.

Create a blog post or video tutorial on a reinforcement learning topic

Show steps

Creating a blog post or video tutorial on a reinforcement learning topic will help students to solidify their understanding of the material. It will also help them to develop their communication skills.

Browse courses on Reinforcement Learning

Show steps

Choose a reinforcement learning topic that you are interested in.
Research the topic and gather information from reliable sources.
Organize your thoughts and create an outline for your blog post or video tutorial.
Write or record your blog post or video tutorial.
Proofread your work and make sure it is clear and concise.

Career center

Learners who complete Fundamentals of Reinforcement Learning will develop knowledge and skills that may be useful to these careers:

Machine Learning Engineer

Machine Learning Engineers design, develop, and maintain machine learning systems. They work on a variety of tasks, including data collection, feature engineering, and model training. Reinforcement Learning Fundamentals will introduce you to the basic concepts and algorithms of reinforcement learning, which is a type of machine learning that allows agents to learn how to behave in an environment by trial and error. This course may be useful for Machine Learning Engineers who want to learn more about reinforcement learning and how to apply it to their work.

See salaries and explore the career path for Machine Learning Engineer

Artificial Intelligence Engineer

Artificial Intelligence Engineers design, develop, and maintain artificial intelligence systems. They work on a variety of tasks, including natural language processing, computer vision, and machine learning. Reinforcement Learning Fundamentals will introduce you to the basic concepts and algorithms of reinforcement learning, which is a type of AI that allows agents to learn how to behave in an environment by trial and error. This course may be useful for Artificial Intelligence Engineers who want to learn more about reinforcement learning and how to apply it to their work.

See salaries and explore the career path for Artificial Intelligence Engineer

Business Analyst

Business Analysts use data and analytics to help businesses make better decisions. They work on a variety of tasks, including process improvement, risk assessment, and financial planning. Reinforcement Learning Fundamentals will introduce you to the basic concepts and algorithms of reinforcement learning, which is a type of AI that allows agents to learn how to behave in an environment by trial and error. This course may be useful for Business Analysts who want to learn more about reinforcement learning and how to apply it to their work.

See salaries and explore the career path for Business Analyst

Operations Research Analyst

Operations Research Analysts use mathematical and analytical techniques to help businesses make better decisions. They work on a variety of tasks, including forecasting, optimization, and simulation. Reinforcement Learning Fundamentals will introduce you to the basic concepts and algorithms of reinforcement learning, which is a type of AI that allows agents to learn how to behave in an environment by trial and error. This course may be useful for Operations Research Analysts who want to learn more about reinforcement learning and how to apply it to their work.

See salaries and explore the career path for Operations Research Analyst

Statistician

Statisticians collect, analyze, and interpret data to help businesses make better decisions. They work on a variety of tasks, including data mining, statistical modeling, and machine learning. Reinforcement Learning Fundamentals will introduce you to the basic concepts and algorithms of reinforcement learning, which is a type of machine learning that allows agents to learn how to behave in an environment by trial and error. This course may be useful for Statisticians who want to learn more about reinforcement learning and how to apply it to their work.

See salaries and explore the career path for Statistician

Software Engineer

Software Engineers design, develop, and maintain software systems. They work on a variety of tasks, including requirements gathering, design, coding, and testing. Reinforcement Learning Fundamentals will introduce you to the basic concepts and algorithms of reinforcement learning, which is a type of AI that allows agents to learn how to behave in an environment by trial and error. This course may be useful for Software Engineers who want to learn more about reinforcement learning and how to apply it to their work.

See salaries and explore the career path for Software Engineer

Project Manager

Project Managers are responsible for the planning, execution, and delivery of projects. They work on a variety of tasks, including project planning, risk management, and stakeholder communication. Reinforcement Learning Fundamentals will introduce you to the basic concepts and algorithms of reinforcement learning, which is a type of AI that allows agents to learn how to behave in an environment by trial and error. This course may be useful for Project Managers who want to learn more about reinforcement learning and how to apply it to their work.

See salaries and explore the career path for Project Manager

Product Manager

Product Managers are responsible for the development and management of products. They work on a variety of tasks, including product design, marketing, and sales. Reinforcement Learning Fundamentals will introduce you to the basic concepts and algorithms of reinforcement learning, which is a type of AI that allows agents to learn how to behave in an environment by trial and error. This course may be useful for Product Managers who want to learn more about reinforcement learning and how to apply it to their work.

See salaries and explore the career path for Product Manager

Consultant

Consultants provide advice and guidance to businesses on a variety of topics. They work on a variety of tasks, including business strategy, process improvement, and financial planning. Reinforcement Learning Fundamentals will introduce you to the basic concepts and algorithms of reinforcement learning, which is a type of AI that allows agents to learn how to behave in an environment by trial and error. This course may be useful for Consultants who want to learn more about reinforcement learning and how to apply it to their work.

See salaries and explore the career path for Consultant

Quantitative Analyst

Quantitative Analysts use mathematical and statistical techniques to analyze financial data and make investment decisions. They work on a variety of tasks, including risk assessment, portfolio optimization, and trading strategy development. Reinforcement Learning Fundamentals will introduce you to the basic concepts and algorithms of reinforcement learning, which is a type of AI that allows agents to learn how to behave in an environment by trial and error. This course may be useful for Quantitative Analysts who want to learn more about reinforcement learning and how to apply it to their work.

See salaries and explore the career path for Quantitative Analyst

Actuary

Actuaries use mathematical and statistical techniques to assess risk and uncertainty. They work on a variety of tasks, including insurance pricing, pension plan design, and financial risk management. Reinforcement Learning Fundamentals will introduce you to the basic concepts and algorithms of reinforcement learning, which is a type of AI that allows agents to learn how to behave in an environment by trial and error. This course may be useful for Actuaries who want to learn more about reinforcement learning and how to apply it to their work.

See salaries and explore the career path for Actuary

Entrepreneur

Entrepreneurs start and run their own businesses. They work on a variety of tasks, including product development, marketing, and sales. Reinforcement Learning Fundamentals will introduce you to the basic concepts and algorithms of reinforcement learning, which is a type of AI that allows agents to learn how to behave in an environment by trial and error. This course may be useful for Entrepreneurs who want to learn more about reinforcement learning and how to apply it to their work.

See salaries and explore the career path for Entrepreneur

Researcher

Researchers conduct scientific research to advance our understanding of the world. They work on a variety of topics, including medicine, physics, and computer science. Reinforcement Learning Fundamentals will introduce you to the basic concepts and algorithms of reinforcement learning, which is a type of AI that allows agents to learn how to behave in an environment by trial and error. This course may be useful for Researchers who want to learn more about reinforcement learning and how to apply it to their work.

See salaries and explore the career path for Researcher

Data Analyst

Data Analysts collect, analyze, and interpret data to help businesses make better decisions. They work on a variety of tasks, including data mining, statistical modeling, and machine learning. Reinforcement Learning Fundamentals will introduce you to the basic concepts and algorithms of reinforcement learning, which is a type of machine learning that allows agents to learn how to behave in an environment by trial and error. This course may be useful for Data Analysts who want to learn more about reinforcement learning and how to apply it to their work.

See salaries and explore the career path for Data Analyst

Data Scientist

Data Scientists collect, analyze, and interpret data to help businesses make better decisions. They work on a variety of tasks, including data mining, statistical modeling, and machine learning. Reinforcement Learning Fundamentals will introduce you to the basic concepts and algorithms of reinforcement learning, which is a type of machine learning that allows agents to learn how to behave in an environment by trial and error. This course may be useful for Data Scientists who want to learn more about reinforcement learning and how to apply it to their work.

See salaries and explore the career path for Data Scientist