We may earn an affiliate commission when you visit our partners.
Decision Making and Reinforcement Learning
Tony Dear
Tony Dear
This course is an introduction to sequential decision making and reinforcement learning. We start with a discussion of utility theory to learn how preferences can be represented and modeled for decision making. We first model simple decision problems as multi-armed bandit problems in and discuss several approaches to evaluate feedback. We will then model decision problems as finite Markov decision processes (MDPs), and discuss their solutions via dynamic programming algorithms. We touch on the notion of partial observability in real problems, modeled by POMDPs and then solved by online planning methods. Finally, we introduce the reinforcement learning problem and discuss two paradigms: Monte Carlo methods and temporal difference learning. We conclude the course by noting how the two paradigms lie on a spectrum of n-step temporal difference methods. An emphasis on algorithms and examples will be a key part of this course.
Register for this course and see more details by visiting:
OpenCourser.com/course/zq6qm4/decision
What's inside
Syllabus
Decision Making and Utility Theory
Welcome to Decision Making and Reinforcement Learning! During this week, Professor Tony Dear provides an overview of the course. You will also view guidelines to support your learning journey towards modeling sequential decision problems and implementing reinforcement learning algorithms.
Read more
Syllabus
Decision Making and Utility Theory
Welcome to Decision Making and Reinforcement Learning! During this week, Professor Tony Dear provides an overview of the course. You will also view guidelines to support your learning journey towards modeling sequential decision problems and implementing reinforcement learning algorithms.
Read more
Bandit Problems
Welcome to week 2! This week, we will learn about multi-armed bandit problems, a type of optimization problem in which the algorithm balances exploration and exploitation to maximize rewards. Topics include action values and sample averaging estimation, 𝜀-greedy action selection, and the upper confidence bound. You could post in the discussion forum if you need assistance on the quiz and assignment.
Markov Decision Processes
Welcome to week 3! This week, we will focus on the basics of the Markov decision process, including rewards, utilities, discounting, policies, value functions, and Bellman equations. You will model sequential decision problems, understand the impact of rewards and discount factors on outcomes, define policies and value functions, and write Bellman equations for optimal solutions. You could post in the discussion forum if you need assistance on the quiz and assignment.
Dynamic Programming
Welcome to week 4! This week, we will cover dynamic programming algorithms for solving Markov decision processes (MDPs). Topics include value iteration and policy iteration, nonlinear Bellman equations, complexity and convergence, and a comparison of the two approaches.You could post in the discussion forum if you need assistance on the quiz and assignment.
Partially Observable Markov Decision Processes
Welcome to week 5! This week, we will go through topics on partial observability and POMDPs, belief states, representation as belief MDPs, and online planning in MDPs and POMDPs. You will also apply your knowledge to update the belief state and employ a belief transition function to calculate state values. You could post in the discussion forum if you need assistance on the quiz and assignment.
Monte Carlo Methods
Welcome to week 6! This week, we will introduce Monte Carlo methods, and cover topics related to state value estimation using sample averaging and Monte Carlo prediction, state-action values and epsilon-greedy policies, and importance sampling for off-policy vs on-policy Monte Carlo control. You will learn to estimate state values, state-action values, use importance sampling, and implement off-policy Monte Carlo control for optimal policy learning. You could post in the discussion forum if you need assistance on the quiz and assignment.
Temporal-Difference Learning
Welcome to week 7! This week, we will cover topics related to temporal difference learning for prediction, TD batch methods, SARSA for on-policy control, and Q-learning for off-policy control. You will learn to implement TD prediction, TD batch and offline methods, SARSA and Q-learning, and compare on-policy vs off-policy TD learning. You will then apply your knowledge in solving a Tic-tac-toe programming assignment.You could post in the discussion forum if you need assistance on the quiz and assignment.
Reinforcement Learning - Generalization
Welcome to week 8! This module covers n-step temporal difference prediction, n-step SARSA (on-policy and off-policy), model-based RL with Dyna-Q, and function approximation. You will be prepared to implement n-step TD learning, n-step SARSA, Dyna-Q for model-based learning, and use function approximation for reinforcement learning. You will apply your knowledge in the Frozen Lake programming environment. You could post in the discussion forum if you need assistance on the quiz and assignment.
Good to know
Good to know
Know what's good ,
what to watch for , and
possible dealbreakers
Introduces students to decision making theory, which is foundational in computer science and data analysis
Emphasizes practical applications, including detailed walkthroughs of reinforcement learning algorithms such as Monte Carlo and temporal difference methods
Offers learners hands-on experience in implementing reinforcement learning algorithms in assignments, providing valuable practical experience
Provides a gradual introduction to the material, starting with simpler concepts like **Markov decision processes** and gradually progressing to more complex ones like **reinforcement learning**
Suitable for motivated students with some foundational knowledge in calculus and linear algebra
Requires learners to actively engage with the material and complete assignments, which may not be suitable for everyone
Save this course
Save Decision Making and Reinforcement Learning to your list so you can find it easily later:
Save
Save
Activities
Be better prepared
before
your course. Deepen your understanding
during
and
after
it. Supplement your coursework and achieve mastery of the topics covered
in Decision Making and Reinforcement Learning with these
activities:
Create a course notes summary
Show steps
Enhance understanding and retention by organizing and summarizing course materials, including notes, assignments, and quizzes.
Show steps
-
Review lecture notes, assignments, and quizzes.
-
Create a concise summary document that highlights key concepts, formulas, and examples.
Brush up on linear algebra
Show steps
Strengthen foundational knowledge in linear algebra to support understanding of reinforcement learning algorithms.
Browse courses on
Linear Algebra
Show steps
-
Review linear algebra concepts from textbooks or online resources.
-
Solve practice problems to reinforce understanding.
Review probability theory basics
Show steps
Reinforce understanding of basic probability theory concepts to strengthen foundational knowledge for this course.
View
Probability & Statistics for Engineers &...
on Amazon
Show steps
-
Read through chapters 1-3 of the book.
-
Solve the practice problems at the end of each chapter.
Five other activities
Expand to see all activities and additional details
Show all eight activities
Participate in a study group
Show steps
Foster collaborative learning and reinforce concepts through discussions and problem-solving with peers.
Show steps
-
Join or form a study group with other course participants.
-
Meet regularly to discuss course material, solve problems, and share insights.
Learn about decision making under uncertainty
Show steps
Extend understanding of how to make informed decisions in uncertain environments, a core concept in this course.
Browse courses on
Decision Making
Show steps
-
Follow the tutorials on 'Decision Making Under Uncertainty' from Coursera or edX.
-
Complete the practice exercises provided in the tutorials.
Attend a machine learning workshop
Show steps
Gain exposure to practical applications and recent advancements in reinforcement learning through a workshop.
Browse courses on
Machine Learning
Show steps
-
Identify and register for a relevant machine learning workshop.
-
Actively participate in the workshop, taking notes and asking questions.
Practice solving MDPs
Show steps
Improve problem-solving skills in the context of Markov decision processes, a key technique covered in this course.
Browse courses on
Markov Decision Processes
Show steps
-
Solve the practice problems on MDPs from the textbook or online resources.
-
Implement MDP algorithms in a programming language of your choice.
Create a reinforcement learning project
Show steps
Apply reinforcement learning concepts to a real-world problem, solidifying understanding and enhancing practical skills.
Browse courses on
Reinforcement Learning
Show steps
-
Choose a reinforcement learning problem to work on.
-
Design and implement a reinforcement learning algorithm to solve the problem.
-
Evaluate the performance of the algorithm and write a report summarizing your findings.
Create a course notes summary
Show steps
Enhance understanding and retention by organizing and summarizing course materials, including notes, assignments, and quizzes.
Show steps
Show steps
Show steps
- Review lecture notes, assignments, and quizzes.
- Create a concise summary document that highlights key concepts, formulas, and examples.
Brush up on linear algebra
Show steps
Strengthen foundational knowledge in linear algebra to support understanding of reinforcement learning algorithms.
Browse courses on
Linear Algebra
Show steps
Show steps
Show steps
- Review linear algebra concepts from textbooks or online resources.
- Solve practice problems to reinforce understanding.
Review probability theory basics
Show steps
Reinforce understanding of basic probability theory concepts to strengthen foundational knowledge for this course.
View
Probability & Statistics for Engineers &...
on Amazon
Show steps
Show steps
Show steps
- Read through chapters 1-3 of the book.
- Solve the practice problems at the end of each chapter.
Five other activities
Expand to see all activities and additional details
Show all eight activities
Participate in a study group
Show steps
Foster collaborative learning and reinforce concepts through discussions and problem-solving with peers.
Show steps
Show steps
Show steps
- Join or form a study group with other course participants.
- Meet regularly to discuss course material, solve problems, and share insights.
Learn about decision making under uncertainty
Show steps
Extend understanding of how to make informed decisions in uncertain environments, a core concept in this course.
Browse courses on
Decision Making
Show steps
Show steps
Show steps
- Follow the tutorials on 'Decision Making Under Uncertainty' from Coursera or edX.
- Complete the practice exercises provided in the tutorials.
Attend a machine learning workshop
Show steps
Gain exposure to practical applications and recent advancements in reinforcement learning through a workshop.
Browse courses on
Machine Learning
Show steps
Show steps
Show steps
- Identify and register for a relevant machine learning workshop.
- Actively participate in the workshop, taking notes and asking questions.
Practice solving MDPs
Show steps
Improve problem-solving skills in the context of Markov decision processes, a key technique covered in this course.
Browse courses on
Markov Decision Processes
Show steps
Show steps
Show steps
- Solve the practice problems on MDPs from the textbook or online resources.
- Implement MDP algorithms in a programming language of your choice.
Create a reinforcement learning project
Show steps
Apply reinforcement learning concepts to a real-world problem, solidifying understanding and enhancing practical skills.
Browse courses on
Reinforcement Learning
Show steps
Show steps
Show steps
- Choose a reinforcement learning problem to work on.
- Design and implement a reinforcement learning algorithm to solve the problem.
- Evaluate the performance of the algorithm and write a report summarizing your findings.
Career center
Learners who complete Decision Making and Reinforcement Learning will develop knowledge and skills
that may be useful to these careers:
Data Scientist
Data Scientist
Data Scientists leverage the power of data through computation to effect business outcomes. Their primary responsibilities involve collecting, cleaning, and analyzing vast amount of data to uncover hidden patterns and provide insights which guide business decisions. Every industry increasingly relies on the expertise of Data Scientists in an era of digital transformation. Decision Making and Reinforcement learning help data scientists make informed decisions about the collection and analysis of data.
Operations Research Analyst
Operations Research Analyst
Operations Research Analysts utilize mathematical and analytical methods to solve complex problems in business and industry. Their work involves building mathematical models to simulate and analyze systems, and they use these models to develop solutions that improve efficiency, productivity and profitability. A strong foundation in Decision Making and Reinforcement Learning is highly valued by employers as it helps build a strong foundation in modeling complex systems and decision-making under uncertainty.
Financial Analyst
Financial Analyst
Financial Analysts provide valuable insights and recommendations to businesses and investors. Their primary responsibilities include analyzing financial data and making recommendations on investment opportunities, business plans, and financial strategies. Decision Making and Reinforcement Learning will provide you with a strong foundation in financial modeling and decision making under uncertainty, which are essential skills for success in this role.
Business Analyst
Business Analyst
Business Analysts play a crucial role in bridging the gap between business and technology by understanding business needs and translating them into technical requirements. They work closely with stakeholders to gather and analyze requirements, and develop solutions that meet the needs of the business. Decision Making and Reinforcement Learning can provide you with a solid foundation in understanding business processes, modeling complex systems, and making data-driven decisions, which are essential skills for success in this role.
Management Consultant
Management Consultant
Management Consultants provide advisory services to businesses to help them improve their performance. They work with clients to identify and solve problems, develop strategies, and implement solutions. Decision Making and Reinforcement Learning can provide you with a strong foundation in problem solving, decision making, and analytical thinking, which are essential skills for success in this role.
Software Engineer
Software Engineer
Software Engineers design, develop, and maintain software systems. They work on a variety of projects, from small personal apps to large enterprise systems. Software Engineers with a background in Decision Making and Reinforcement Learning are highly sought after by employers, as they have a strong foundation in modeling complex systems and making data-driven decisions.
Quantitative Analyst
Quantitative Analyst
Quantitative Analysts use mathematical and statistical methods to analyze and interpret data. They work in a variety of industries, including finance, healthcare, and technology. Quantitative Analysts with a background in Decision Making and Reinforcement Learning are highly valued by employers, as they have a strong foundation in modeling complex systems and making data-driven decisions.
Data Engineer
Data Engineer
Data Engineers design, build, and maintain the infrastructure that stores and processes data. They work with a variety of data sources, including databases, data warehouses, and data lakes. Data Engineers with a background in Decision Making and Reinforcement Learning are highly sought after by employers, as they have a strong foundation in modeling complex systems and making data-driven decisions.
Product Manager
Product Manager
Product Managers are responsible for the development and launch of new products. They work with a variety of stakeholders, including engineers, designers, and marketing teams, to bring products to market. Product Managers with a background in Decision Making and Reinforcement Learning are highly valued by employers, as they have a strong foundation in understanding customer needs and making data-driven decisions.
Machine Learning Engineer
Machine Learning Engineer
Machine Learning Engineers design, develop, and deploy machine learning models. They work on a variety of projects, from developing new algorithms to deploying models into production. Machine Learning Engineers with a background in Decision Making and Reinforcement Learning are highly sought after by employers, as they have a strong foundation in modeling complex systems and making data-driven decisions.
Statistician
Statistician
Statisticians collect, analyze, and interpret data. They work in a variety of industries, including healthcare, finance, and education. Statisticians with a background in Decision Making and Reinforcement Learning are highly valued by employers, as they have a strong foundation in modeling complex systems and making data-driven decisions.
Actuary
Actuary
Actuaries use mathematical and statistical methods to assess risk and uncertainty. They work in a variety of industries, including insurance, finance, and healthcare. Actuaries with a background in Decision Making and Reinforcement Learning are highly valued by employers, as they have a strong foundation in modeling complex systems and making data-driven decisions.
Risk Analyst
Risk Analyst
Risk Analysts identify, assess, and manage risks. They work in a variety of industries, including finance, healthcare, and technology. Risk Analysts with a background in Decision Making and Reinforcement Learning are highly valued by employers, as they have a strong foundation in modeling complex systems and making data-driven decisions.
Operations Manager
Operations Manager
Operations Managers plan, organize, and direct the operations of an organization. They work in a variety of industries, including manufacturing, healthcare, and retail. Operations Managers with a background in Decision Making and Reinforcement Learning are highly valued by employers, as they have a strong foundation in modeling complex systems and making data-driven decisions.
Project Manager
Project Manager
Project Managers plan, organize, and execute projects. They work in a variety of industries, including construction, software development, and healthcare. Project Managers with a background in Decision Making and Reinforcement Learning are highly valued by employers, as they have a strong foundation in modeling complex systems and making data-driven decisions.
For more career information including salaries, visit:
OpenCourser.com/course/zq6qm4/decision
Reading list
We've selected eight books
that we think will supplement your
learning. Use these to
develop background knowledge, enrich your coursework, and gain a
deeper understanding of the topics covered in
Decision Making and Reinforcement Learning.
Provides a comprehensive introduction to the field of reinforcement learning, covering both the theoretical foundations and practical applications. It valuable resource for anyone interested in learning more about this important topic.
Provides a comprehensive overview of Markov decision processes.
Provides an introduction to the field of stochastic dynamic programming.
Provides an introduction to adaptive dynamic programming and reinforcement learning, two powerful tools for modeling and solving sequential decision making problems. It valuable resource for anyone interested in learning more about this important topic.
Provides a comprehensive introduction to Markov decision processes, a powerful tool for modeling and solving sequential decision making problems. It valuable resource for anyone interested in learning more about this important topic.
Classic work on reinforcement learning and Markov decision processes. It valuable resource for anyone interested in learning more about this important topic.
Provides a comprehensive introduction to decision making under uncertainty, a fundamental problem in many fields. It valuable resource for anyone interested in learning more about this important topic.
Provides an introduction to bandit algorithms, a powerful tool for website optimization. It valuable resource for anyone interested in learning more about this important topic.
For more information about how these books relate to this course, visit:
OpenCourser.com/course/zq6qm4/decision
Share
Similar courses
Here are nine courses similar to
Decision Making and Reinforcement Learning.
Introduction to Reinforcement Learning in Python
OpenCourser.com/course/9uoty0/introduction
Introduction to Reinforcement Learning in Python
Most relevant
Understanding Algorithms for Reinforcement Learning
OpenCourser.com/course/4hbndm/understanding
Understanding Algorithms for Reinforcement Learning
Most relevant
Sample-based Learning Methods
OpenCourser.com/course/mk6kp1/sample
Sample-based Learning Methods
Most relevant
Fundamentals of Reinforcement Learning
OpenCourser.com/course/svbkm5/fundamentals
Fundamentals of Reinforcement Learning
Most relevant
Decision-Making for Autonomous Systems
OpenCourser.com/course/9md05l/decision
Decision-Making for Autonomous Systems
Most relevant
303: Algorith
OpenCourser.com/course/ted56s/sayloracademy
303: Algorith
Most relevant
Overview of Advanced Methods of Reinforcement Learning in Finance
OpenCourser.com/course/ezzvtp/overview
Overview of Advanced Methods of Reinforcement Learning in...
Most relevant
Machine Learning with Python: from Linear Models to Deep Learning
OpenCourser.com/course/lpt9e1/machine
Machine Learning with Python: from Linear Models to Deep...
Reinforcement Learning: Qwik Start
OpenCourser.com/course/ztz2qn/reinforcement
Reinforcement Learning: Qwik Start
Effort
12 weeks of study, 7-10 hours/week
Level
Intermediate
Via
Coursera
Institution
Columbia University
Instructor
Tony Dear
Language
English
Transcripts
Español
Français
Português
Deutsch
Italiano
中文
العربية
한국어
日本語
हिंदी
Nederlands
Svenska
Pусский
Türkçe
Polski
Ελληνικά
українська мова
magyar nyelv
Bahasa Indonesia
ไทย
қазақша
اُردُو,
فارسی
বাংলা
پښتو
آذربایجان دیلی
Good to know
Introduces students to decision making theory, which is foundational in computer science and data analysis
Emphasizes practical applications, including detailed walkthroughs of reinforcement learning algorithms such as Monte Carlo and temporal difference methods
Offers learners hands-on experience in implementing reinforcement learning algorithms in assignments, providing valuable practical experience
Provides a gradual introduction to the material, starting with simpler concepts like **Markov decision processes** and gradually progressing to more complex ones like **reinforcement learning**
Suitable for motivated students with some foundational knowledge in calculus and linear algebra
Requires learners to actively engage with the material and complete assignments, which may not be suitable for everyone
Our mission
OpenCourser helps millions of learners each year. People visit us to learn workspace skills, ace their exams, and nurture their curiosity.
Our extensive catalog contains over 50,000 courses and twice as many books. Browse by search, by topic, or even by career interests. We'll match you to the right resources quickly.
Find this site helpful? Tell a friend about us.
Affiliate disclosure
We're supported by our community of learners. When you purchase or subscribe to courses and programs or purchase books, we may earn a commission from our partners.
Your purchases help us maintain our catalog and keep our servers humming without ads.
Thank you for supporting OpenCourser.
© 2016 - 2024 OpenCourser