We may earn an affiliate commission when you visit our partners.
Tony Dear
This course is an introduction to sequential decision making and reinforcement learning. We start with a discussion of utility theory to learn how preferences can be represented and modeled for decision making. We first model simple decision problems as multi-armed bandit problems in and discuss several approaches to evaluate feedback. We will then model decision problems as finite Markov decision processes (MDPs), and discuss their solutions via dynamic programming algorithms. We touch on the notion of partial observability in real problems, modeled by POMDPs and then solved by online planning methods. Finally, we introduce the reinforcement learning problem and discuss two paradigms: Monte Carlo methods and temporal difference learning. We conclude the course by noting how the two paradigms lie on a spectrum of n-step temporal difference methods. An emphasis on algorithms and examples will be a key part of this course.
Register for this course and see more details by visiting:
OpenCourser.com/course/zq6qm4/decision
Two deals to help you save
Save money when you learn. Here are two deals and offers that may be relevant to this course.
All coupon codes, vouchers, and discounts are applied automatically unless otherwise noted.
Valid until July 14
Coursera Plus Annual
Work toward what's next for your career with access to 10,000+ programs. Discover emerging skills and save.
Up to
40%
off
Get offer
Valid until August 30
Google AI App Builder
Learn how to use Gemini API and API Studio with a three-course series from Google DeepMind
Take
100%
off
Get offer
What's inside
Syllabus
Decision Making and Utility Theory
Welcome to Decision Making and Reinforcement Learning! During this week, Professor Tony Dear provides an overview of the course. You will also view guidelines to support your learning journey towards modeling sequential decision problems and implementing reinforcement learning algorithms.
Read more
Syllabus
Decision Making and Utility Theory
Welcome to Decision Making and Reinforcement Learning! During this week, Professor Tony Dear provides an overview of the course. You will also view guidelines to support your learning journey towards modeling sequential decision problems and implementing reinforcement learning algorithms.
Read more
Traffic lights
Traffic lights
Read about what's good
what should give you pause and
possible dealbreakers
Introduces students to decision making theory, which is foundational in computer science and data analysis
Emphasizes practical applications, including detailed walkthroughs of reinforcement learning algorithms such as Monte Carlo and temporal difference methods
Offers learners hands-on experience in implementing reinforcement learning algorithms in assignments, providing valuable practical experience
Provides a gradual introduction to the material, starting with simpler concepts like **Markov decision processes** and gradually progressing to more complex ones like **reinforcement learning**
Suitable for motivated students with some foundational knowledge in calculus and linear algebra
Requires learners to actively engage with the material and complete assignments, which may not be suitable for everyone
Save this course
Create your own learning path.
Save this course to your list so you can find it easily later.
Save
Save
Reviews summary
Comprehensive reinforcement learning foundation
According to learners, this course provides a strong theoretical foundation in sequential decision-making and reinforcement learning. Students consistently praise the clear and engaging lectures from the instructor, finding them effective in demystifying complex topics like MDPs and temporal difference learning. The hands-on programming assignments are a significant highlight, described as instrumental in solidifying algorithmic understanding. While the course is largely positive, some note that it can be challenging for those without a solid math background, and a few wish for more advanced practical applications. Overall, it's considered an excellent introduction for technically inclined learners.
Leans more on theory with foundational practical examples.
"The course is definitely heavy on the theoretical aspects of RL, which I appreciated."
"While the assignments are good, I would have liked more extensive real-world projects beyond the foundational ones."
"It builds a great theoretical base, but don't expect deep dives into advanced practical implementations."
Instructor explains complex topics with great clarity.
"Professor Dear makes even the most difficult concepts accessible and easy to follow."
"The instructor's explanations of dynamic programming and TD learning were particularly clear."
"I found the lectures engaging and well-structured, which kept me motivated throughout."
Hands-on exercises deepen understanding and application.
"The programming assignments, especially Tic-tac-toe and Frozen Lake, were invaluable for applying what I learned."
"I loved how the practical exercises helped solidify my grasp of Monte Carlo and Q-learning."
"Completing the labs truly brought the theoretical concepts to life for me."
Builds a robust understanding of core RL concepts.
"The lectures provide a solid theoretical groundwork for understanding complex reinforcement learning algorithms."
"I really appreciate the depth of coverage on Markov Decision Processes and utility theory."
"This course helped me build a fundamental understanding of RL principles and decision-making."
Requires solid math and programming background for success.
"I found the course quite challenging, and I think a stronger background in linear algebra and probability is essential."
"The pace can be very fast if you're not already comfortable with mathematical notation and Python coding."
"This course is great, but be prepared with solid math and CS fundamentals; it's not for beginners in those areas."
Activities
Be better prepared
before
your course. Deepen your understanding
during
and
after
it. Supplement your coursework and achieve mastery of the topics covered
in Decision Making and Reinforcement Learning with these
activities:
Create a course notes summary
Show steps
Enhance understanding and retention by organizing and summarizing course materials, including notes, assignments, and quizzes.
Show steps
-
Review lecture notes, assignments, and quizzes.
-
Create a concise summary document that highlights key concepts, formulas, and examples.
Brush up on linear algebra
Show steps
Strengthen foundational knowledge in linear algebra to support understanding of reinforcement learning algorithms.
Browse courses on
Linear Algebra
Show steps
-
Review linear algebra concepts from textbooks or online resources.
-
Solve practice problems to reinforce understanding.
Review probability theory basics
Show steps
Reinforce understanding of basic probability theory concepts to strengthen foundational knowledge for this course.
View
Probability & Statistics for Engineers &...
on Amazon
Show steps
-
Read through chapters 1-3 of the book.
-
Solve the practice problems at the end of each chapter.
Five other activities
Expand to see all activities and additional details
Show all eight activities
Participate in a study group
Show steps
Foster collaborative learning and reinforce concepts through discussions and problem-solving with peers.
Show steps
-
Join or form a study group with other course participants.
-
Meet regularly to discuss course material, solve problems, and share insights.
Learn about decision making under uncertainty
Show steps
Extend understanding of how to make informed decisions in uncertain environments, a core concept in this course.
Browse courses on
Decision Making
Show steps
-
Follow the tutorials on 'Decision Making Under Uncertainty' from Coursera or edX.
-
Complete the practice exercises provided in the tutorials.
Attend a machine learning workshop
Show steps
Gain exposure to practical applications and recent advancements in reinforcement learning through a workshop.
Browse courses on
Machine Learning
Show steps
-
Identify and register for a relevant machine learning workshop.
-
Actively participate in the workshop, taking notes and asking questions.
Practice solving MDPs
Show steps
Improve problem-solving skills in the context of Markov decision processes, a key technique covered in this course.
Browse courses on
Markov Decision Processes
Show steps
-
Solve the practice problems on MDPs from the textbook or online resources.
-
Implement MDP algorithms in a programming language of your choice.
Create a reinforcement learning project
Show steps
Apply reinforcement learning concepts to a real-world problem, solidifying understanding and enhancing practical skills.
Browse courses on
Reinforcement Learning
Show steps
-
Choose a reinforcement learning problem to work on.
-
Design and implement a reinforcement learning algorithm to solve the problem.
-
Evaluate the performance of the algorithm and write a report summarizing your findings.
Create a course notes summary
Show steps
Enhance understanding and retention by organizing and summarizing course materials, including notes, assignments, and quizzes.
Show steps
Show steps
Show steps
- Review lecture notes, assignments, and quizzes.
- Create a concise summary document that highlights key concepts, formulas, and examples.
Brush up on linear algebra
Show steps
Strengthen foundational knowledge in linear algebra to support understanding of reinforcement learning algorithms.
Browse courses on
Linear Algebra
Show steps
Show steps
Show steps
- Review linear algebra concepts from textbooks or online resources.
- Solve practice problems to reinforce understanding.
Review probability theory basics
Show steps
Reinforce understanding of basic probability theory concepts to strengthen foundational knowledge for this course.
View
Probability & Statistics for Engineers &...
on Amazon
Show steps
Show steps
Show steps
- Read through chapters 1-3 of the book.
- Solve the practice problems at the end of each chapter.
Five other activities
Expand to see all activities and additional details
Show all eight activities
Participate in a study group
Show steps
Foster collaborative learning and reinforce concepts through discussions and problem-solving with peers.
Show steps
Show steps
Show steps
- Join or form a study group with other course participants.
- Meet regularly to discuss course material, solve problems, and share insights.
Learn about decision making under uncertainty
Show steps
Extend understanding of how to make informed decisions in uncertain environments, a core concept in this course.
Browse courses on
Decision Making
Show steps
Show steps
Show steps
- Follow the tutorials on 'Decision Making Under Uncertainty' from Coursera or edX.
- Complete the practice exercises provided in the tutorials.
Attend a machine learning workshop
Show steps
Gain exposure to practical applications and recent advancements in reinforcement learning through a workshop.
Browse courses on
Machine Learning
Show steps
Show steps
Show steps
- Identify and register for a relevant machine learning workshop.
- Actively participate in the workshop, taking notes and asking questions.
Practice solving MDPs
Show steps
Improve problem-solving skills in the context of Markov decision processes, a key technique covered in this course.
Browse courses on
Markov Decision Processes
Show steps
Show steps
Show steps
- Solve the practice problems on MDPs from the textbook or online resources.
- Implement MDP algorithms in a programming language of your choice.
Create a reinforcement learning project
Show steps
Apply reinforcement learning concepts to a real-world problem, solidifying understanding and enhancing practical skills.
Browse courses on
Reinforcement Learning
Show steps
Show steps
Show steps
- Choose a reinforcement learning problem to work on.
- Design and implement a reinforcement learning algorithm to solve the problem.
- Evaluate the performance of the algorithm and write a report summarizing your findings.
Career center
Learners who complete Decision Making and Reinforcement Learning will develop knowledge and skills
that may be useful to these careers:
Data Scientist
Data Scientist
Data Scientists leverage the power of data through computation to effect business outcomes. Their primary responsibilities involve collecting, cleaning, and analyzing vast amount of data to uncover hidden patterns and provide insights which guide business decisions. Every industry increasingly relies on the expertise of Data Scientists in an era of digital transformation. Decision Making and Reinforcement learning help data scientists make informed decisions about the collection and analysis of data.
Operations Research Analyst
Operations Research Analyst
Operations Research Analysts utilize mathematical and analytical methods to solve complex problems in business and industry. Their work involves building mathematical models to simulate and analyze systems, and they use these models to develop solutions that improve efficiency, productivity and profitability. A strong foundation in Decision Making and Reinforcement Learning is highly valued by employers as it helps build a strong foundation in modeling complex systems and decision-making under uncertainty.
Financial Analyst
Financial Analyst
Financial Analysts provide valuable insights and recommendations to businesses and investors. Their primary responsibilities include analyzing financial data and making recommendations on investment opportunities, business plans, and financial strategies. Decision Making and Reinforcement Learning will provide you with a strong foundation in financial modeling and decision making under uncertainty, which are essential skills for success in this role.
Business Analyst
Business Analyst
Business Analysts play a crucial role in bridging the gap between business and technology by understanding business needs and translating them into technical requirements. They work closely with stakeholders to gather and analyze requirements, and develop solutions that meet the needs of the business. Decision Making and Reinforcement Learning can provide you with a solid foundation in understanding business processes, modeling complex systems, and making data-driven decisions, which are essential skills for success in this role.
Management Consultant
Management Consultant
Management Consultants provide advisory services to businesses to help them improve their performance. They work with clients to identify and solve problems, develop strategies, and implement solutions. Decision Making and Reinforcement Learning can provide you with a strong foundation in problem solving, decision making, and analytical thinking, which are essential skills for success in this role.
Software Engineer
Software Engineer
Software Engineers design, develop, and maintain software systems. They work on a variety of projects, from small personal apps to large enterprise systems. Software Engineers with a background in Decision Making and Reinforcement Learning are highly sought after by employers, as they have a strong foundation in modeling complex systems and making data-driven decisions.
Quantitative Analyst
Quantitative Analyst
Quantitative Analysts use mathematical and statistical methods to analyze and interpret data. They work in a variety of industries, including finance, healthcare, and technology. Quantitative Analysts with a background in Decision Making and Reinforcement Learning are highly valued by employers, as they have a strong foundation in modeling complex systems and making data-driven decisions.
Data Engineer
Data Engineer
Data Engineers design, build, and maintain the infrastructure that stores and processes data. They work with a variety of data sources, including databases, data warehouses, and data lakes. Data Engineers with a background in Decision Making and Reinforcement Learning are highly sought after by employers, as they have a strong foundation in modeling complex systems and making data-driven decisions.
Product Manager
Product Manager
Product Managers are responsible for the development and launch of new products. They work with a variety of stakeholders, including engineers, designers, and marketing teams, to bring products to market. Product Managers with a background in Decision Making and Reinforcement Learning are highly valued by employers, as they have a strong foundation in understanding customer needs and making data-driven decisions.
Machine Learning Engineer
Machine Learning Engineer
Machine Learning Engineers design, develop, and deploy machine learning models. They work on a variety of projects, from developing new algorithms to deploying models into production. Machine Learning Engineers with a background in Decision Making and Reinforcement Learning are highly sought after by employers, as they have a strong foundation in modeling complex systems and making data-driven decisions.
Statistician
Statistician
Statisticians collect, analyze, and interpret data. They work in a variety of industries, including healthcare, finance, and education. Statisticians with a background in Decision Making and Reinforcement Learning are highly valued by employers, as they have a strong foundation in modeling complex systems and making data-driven decisions.
Actuary
Actuary
Actuaries use mathematical and statistical methods to assess risk and uncertainty. They work in a variety of industries, including insurance, finance, and healthcare. Actuaries with a background in Decision Making and Reinforcement Learning are highly valued by employers, as they have a strong foundation in modeling complex systems and making data-driven decisions.
Risk Analyst
Risk Analyst
Risk Analysts identify, assess, and manage risks. They work in a variety of industries, including finance, healthcare, and technology. Risk Analysts with a background in Decision Making and Reinforcement Learning are highly valued by employers, as they have a strong foundation in modeling complex systems and making data-driven decisions.
Operations Manager
Operations Manager
Operations Managers plan, organize, and direct the operations of an organization. They work in a variety of industries, including manufacturing, healthcare, and retail. Operations Managers with a background in Decision Making and Reinforcement Learning are highly valued by employers, as they have a strong foundation in modeling complex systems and making data-driven decisions.
Project Manager
Project Manager
Project Managers plan, organize, and execute projects. They work in a variety of industries, including construction, software development, and healthcare. Project Managers with a background in Decision Making and Reinforcement Learning are highly valued by employers, as they have a strong foundation in modeling complex systems and making data-driven decisions.
For more career information including salaries, visit:
OpenCourser.com/course/zq6qm4/decision
Reading list
We've selected eight books
that we think will supplement your
learning. Use these to
develop background knowledge, enrich your coursework, and gain a
deeper understanding of the topics covered in
Decision Making and Reinforcement Learning.
Provides a comprehensive introduction to the field of reinforcement learning, covering both the theoretical foundations and practical applications. It valuable resource for anyone interested in learning more about this important topic.
Provides a comprehensive overview of Markov decision processes.
Provides an introduction to the field of stochastic dynamic programming.
Provides an introduction to adaptive dynamic programming and reinforcement learning, two powerful tools for modeling and solving sequential decision making problems. It valuable resource for anyone interested in learning more about this important topic.
Provides a comprehensive introduction to Markov decision processes, a powerful tool for modeling and solving sequential decision making problems. It valuable resource for anyone interested in learning more about this important topic.
Classic work on reinforcement learning and Markov decision processes. It valuable resource for anyone interested in learning more about this important topic.
Provides a comprehensive introduction to decision making under uncertainty, a fundamental problem in many fields. It valuable resource for anyone interested in learning more about this important topic.
Provides an introduction to bandit algorithms, a powerful tool for website optimization. It valuable resource for anyone interested in learning more about this important topic.
For more information about how these books relate to this course, visit:
OpenCourser.com/course/zq6qm4/decision
Share
Similar courses
Similar courses are unavailable at this time. Please try again later.
Rating
4.2
Effort
47 hours to complete
Level
Intermediate
Via
Coursera
Institution
Columbia University
Instructor
Tony Dear
Language
English
Transcripts
Español
Français
Português
Deutsch
Italiano
中文
العربية
한국어
日本語
हिंदी
Nederlands
Svenska
Pусский
Türkçe
Polski
Ελληνικά
українська мова
magyar nyelv
Bahasa Indonesia
ไทย
қазақша
اُردُو,
فارسی
বাংলা
پښتو
آذربایجان دیلی
Our mission
OpenCourser helps millions of learners each year. People visit us to learn workspace skills, ace their exams, and nurture their curiosity.
Our extensive catalog contains over 50,000 courses and twice as many books. Browse by search, by topic, or even by career interests. We'll match you to the right resources quickly.
Find this site helpful? Tell a friend about us.
Affiliate disclosure
We're supported by our community of learners. When you purchase or subscribe to courses and programs or purchase books, we may earn a commission from our partners.
Your purchases help us maintain our catalog and keep our servers humming without ads.
Thank you for supporting OpenCourser.
© 2016 - 2025 OpenCourser