We may earn an affiliate commission when you visit our partners.
Course image
Thomas Woolf

Sequential Decisions builds from math and algorithms that can be understood and used by Coursera Students. This course will start from a consideration of the simplest type of data streams and then gradually advance to more complex types of data and more nuanced decisions being made on that data. You will be able to: (a) program optimal decisions for data arriving from known distribution functions, (b) define error bars and nuanced hedges about ongoing data streams to reflect missing data and/or missing knowledge, (c)understand and use the connections from these models to further understand Markov Chains and Markov Processes and how these ideas connect to Reinforcement Learning and (d) Understand better the nuances between time-independent, time-dependent, one-dimensional and multi-dimensional data.

Read more

Sequential Decisions builds from math and algorithms that can be understood and used by Coursera Students. This course will start from a consideration of the simplest type of data streams and then gradually advance to more complex types of data and more nuanced decisions being made on that data. You will be able to: (a) program optimal decisions for data arriving from known distribution functions, (b) define error bars and nuanced hedges about ongoing data streams to reflect missing data and/or missing knowledge, (c)understand and use the connections from these models to further understand Markov Chains and Markov Processes and how these ideas connect to Reinforcement Learning and (d) Understand better the nuances between time-independent, time-dependent, one-dimensional and multi-dimensional data.

The course is aimed at those working with data, this includes both those charged with analyzing the data and those in charge of making decisions based on that data.

Enroll now

What's inside

Syllabus

Wald and Sequential Decisions
This module introduces the class and the approach to teaching it to be used for the next five weeks. We begin with simple sequential data, similar to Wald’s model: data arrives from a distribution and is not time dependent. This can be generative data. We then explore increasingly complex data from distributions collected for health or business reasons. We finish the week with connections to code work and to AI.
Read more

Traffic lights

Read about what's good
what should give you pause
and possible dealbreakers
Builds deep understanding of math and algorithms
Taught by experienced and well-respected instructors
Designed for professionals working with data
Provides a comprehensive study of sequential decisions
Covers a range of topics, from basic concepts to advanced techniques
Prerequisites may be required for some students

Save this course

Create your own learning path. Save this course to your list so you can find it easily later.
Save

Reviews summary

Advanced theory for data science decisions

According to students, this course offers a rigorous and insightful exploration into the theoretical foundations of sequential decision-making in data science. Learners praise its mathematical depth and the instructor's ability to connect complex concepts like Markov Chains and Reinforcement Learning to practical applications. However, a consistent theme is the demanding prerequisite knowledge, particularly in advanced mathematics. While many find the lectures to be clear and detailed, some express a desire for more hands-on coding examples and immediate practical application, noting it can feel overly academic. The course is best suited for those seeking a strong theoretical understanding rather than a quick practical guide.
Instructor is highly knowledgeable and passionate about the subject.
"The instructor is knowledgeable and passionate about the subject matter."
"The professor is exceptional, clearly an expert in the field."
"I appreciate how the professor brilliantly connects theoretical concepts to real-world data science problems."
Provides a rigorous understanding of sequential decision theory.
"This course was an eye-opener. The way Prof. XXX connects theoretical concepts like Markov Chains to practical data science decisions is brilliant."
"I found the modules on Markov Decision Processes particularly insightful, linking directly to reinforcement learning."
"I gained a strong theoretical understanding of data science decisions, covering complex topics like change points and MDPs very well."
Lectures can be dense, fast-paced, requiring multiple re-watches.
"The lectures can be a bit fast-paced, sometimes requiring multiple re-watches to grasp the material."
"I found some parts felt a bit rushed, especially towards the end of the course."
"I had to review the material multiple times, especially if I wasn't fully comfortable with advanced math concepts."
Divisive opinions on the practical application vs. theoretical focus.
"It felt too theoretical and not practical enough; I was looking for something that would help me use data effectively in my job."
"I struggled with the delivery; it felt very academic and less practical for immediate application than I hoped."
"This course has excellent theoretical content, but I wish it had more practical, hands-on coding exercises."
"I found the theoretical concepts incredibly insightful, appreciating the mathematical rigor over immediate application."
Requires a strong background in mathematics and prior knowledge.
"This course is definitely for those with a solid math background and interest in advanced topics."
"The prerequisites are real—I realized I needed to know calculus and linear algebra thoroughly."
"I felt lost quickly because I underestimated the mathematical prerequisites; it's definitely not for beginners."
"The instructor assumed a lot of prior knowledge, which made it challenging."

Activities

Be better prepared before your course. Deepen your understanding during and after it. Supplement your coursework and achieve mastery of the topics covered in Data Science Decisions in Time: Using Data Effectively with these activities:
Review Probability and Linear Algebra
Refreshing your knowledge of probability and linear algebra will provide a solid foundation for understanding the concepts in this course.
Browse courses on Probability
Show steps
  • Review your notes or textbooks on probability and linear algebra.
  • Solve practice problems to test your understanding.
  • Seek help from a tutor or mentor if needed.
Curate Resources on Markov Processes
Compiling resources related to Markov Processes will provide you with a valuable collection for future reference and deeper exploration.
Browse courses on Markov Processes
Show steps
  • Search for online resources on Markov Processes, including articles, tutorials, and videos.
  • Organize the resources into a structured format, such as a document or spreadsheet.
  • Annotate each resource with a brief description and your evaluation of its quality.
Review Pre-Calculus
Review the trigonometric identities and properties of functions.
Browse courses on Pre-Calculus
Show steps
  • Revisit unit circle and special triangle ratios.
  • Refresh properties of sine, cosine, and tangent functions.
  • Examine the graphs of trigonometric functions and their transformations.
  • Practice solving trigonometric equations.
11 other activities
Expand to see all activities and additional details
Show all 14 activities
Industry Expert Mentorship
Connect with experienced professionals to gain insights into real-world applications and best practices in sequential decision-making.
Browse courses on Machine Learning
Show steps
  • Identify potential mentors in relevant fields
  • Reach out and schedule informational interviews
  • Seek guidance on career development and technical challenges
Discuss Sequential Decision Making
Engaging in discussions with peers will help you clarify your understanding of sequential decision making and gain new perspectives.
Show steps
  • Find a study partner or join a study group.
  • Choose a topic related to sequential decision making.
  • Discuss your ideas and insights with your partner or group.
  • Work together to solve problems and answer questions.
Collaborative Study Groups
Enhance understanding and critical thinking by discussing concepts, solving problems, and sharing perspectives with peers.
Browse courses on Markov Processes
Show steps
  • Form a study group with 2-4 other students
  • Set regular meeting times and goals
  • Take turns presenting concepts, leading discussions, and solving problems
Explore Markov Chain Tutorials
Following this series of tutorials will help provide a conceptual understanding of how Markov Chains work and how they can be applied to real-world scenarios.
Browse courses on Markov Chains
Show steps
  • Identify introductory Markov Chain tutorials with high ratings.
  • Follow the instructions and work through the examples.
  • Apply the concepts you learned to solve practice problems.
  • Document the key concepts and techniques you learned.
Code Challenges
Reinforce theoretical concepts and improve algorithmic thinking by solving challenging programming problems.
Show steps
  • Identify key concepts and algorithms
  • Break down the problem into smaller steps
  • Implement solutions and test correctness
Build a Markov Chain Model
Building a Markov Chain model will provide you with hands-on experience and a deeper understanding of how they work.
Browse courses on Markov Chains
Show steps
  • Identify a real-world scenario that can be modeled using a Markov Chain.
  • Create a state transition matrix for your model.
  • Implement the model in a programming language.
  • Analyze the output of your model and draw conclusions.
Interactive Simulations
Develop a deeper understanding of complex concepts by creating interactive simulations that illustrate their behavior.
Browse courses on Markov Chains
Show steps
  • Choose a specific concept or model to simulate
  • Design the simulation interface and logic
  • Implement the simulation using appropriate software tools
Data Analysis Project
Apply sequential decision-making principles to real-world data and present findings in a well-structured report.
Browse courses on Data Analytics
Show steps
  • Identify a suitable dataset and problem statement
  • Develop a decision-making model and algorithm
  • Analyze data and evaluate results
Solve Markov Decision Process Problems
Solving Markov Decision Process problems will strengthen your understanding of the concepts and improve your problem-solving skills.
Browse courses on Markov Decision Processes
Show steps
  • Find a collection of Markov Decision Process practice problems.
  • Solve the problems using the concepts you learned in the course.
  • Analyze your solutions and identify areas for improvement.
  • Document the key concepts and techniques you used.
Study 'Reinforcement Learning: An Introduction'
This book provides a comprehensive overview of reinforcement learning, which builds on the concepts of Markov Decision Processes.
Show steps
  • Read the book thoroughly, taking notes on key concepts.
  • Solve the exercises and practice problems provided in the book.
  • Discuss the book's concepts with peers or a mentor.
Contribute to an Open-Source Markov Chain Library
Contributing to an open-source Markov Chain library will provide you with practical experience and strengthen your understanding of the underlying algorithms.
Browse courses on Markov Chains
Show steps
  • Identify a reputable open-source Markov Chain library.
  • Review the library's documentation and codebase.
  • Identify an area where you can contribute, such as adding a new feature or improving the documentation.
  • Submit a pull request with your changes.

Career center

Learners who complete Data Science Decisions in Time: Using Data Effectively will develop knowledge and skills that may be useful to these careers:

Reading list

We've selected 35 books that we think will supplement your learning. Use these to develop background knowledge, enrich your coursework, and gain a deeper understanding of the topics covered in Data Science Decisions in Time: Using Data Effectively.
Is widely considered the foundational text in reinforcement learning, providing a comprehensive introduction to the field's key ideas and algorithms. It is suitable for gaining a broad understanding and is often used as a primary textbook in academic settings. The second edition includes updated coverage and new topics, making it relevant for both students and professionals seeking a solid theoretical grounding.
A comprehensive and rigorous textbook covering both probability theory and stochastic processes, with extensive material on Markov Chains. is suitable for deepening understanding and is widely used in mathematics and statistics programs at the advanced undergraduate and graduate levels. It provides a solid theoretical foundation and numerous examples.
This forthcoming book focuses on the contemporary and increasingly important topic of Reinforcement Learning from Human Feedback (RLHF). It provides an introduction to the core methods and discusses advanced topics and open questions in this rapidly developing area. It's highly relevant for those interested in the latest advancements in RL.
Provides a comprehensive overview of Markov decision processes (MDPs), including theory, algorithms, and applications. It is especially useful for readers who are interested in the mathematical foundations of MDPs and their use in solving complex decision-making problems.
Offers a practical approach to deep reinforcement learning, focusing on applying modern RL methods to real-world problems. It is valuable for those looking to deepen their understanding by implementing algorithms and working through practical examples. The book covers a wide range of topics and is particularly useful for practitioners.
This widely acclaimed textbook provides a comprehensive introduction to probability models, including a substantial section on Markov Chains. It is an excellent resource for gaining a broad understanding of the fundamental concepts and their applications. Often used as a primary textbook in undergraduate and graduate programs, it helps solidify understanding and serves as a valuable reference.
Uniquely combines the theoretical foundations of deep reinforcement learning with practical implementation in Python. It's a valuable resource for students and practitioners who want to understand both the 'why' and the 'how' of DRL algorithms. It builds from the basics to more complex topics.
Offers a hands-on guide to implementing deep reinforcement learning projects. It covers fundamental concepts and algorithms and progresses to more advanced topics, with an emphasis on practical application. It's ideal for those who learn best by doing and want to quickly apply DRL to real-world scenarios.
Demystifies reinforcement learning with a focus on its applications in finance. It emphasizes the foundational mathematics and provides Python code for implementing models and algorithms relevant to financial trading problems. It's a good resource for those specifically interested in RL in finance.
A comprehensive and classic graduate-level textbook that provides in-depth coverage of a wide range of stochastic processes, with significant sections dedicated to Markov Chains. It is highly valuable for students seeking a thorough theoretical understanding and is often used as a reference in academic settings. While an older publication, its depth and rigor remain relevant.
Offers a comprehensive guide to building RL agents using Python libraries like TensorFlow and RLib. It focuses on solving complex problems with RL and provides practical tips and best practices for professionals. It's a good resource for deepening understanding through implementation.
Focuses on the advanced and contemporary topic of mixing times and convergence of Markov chains, which is crucial in areas like MCMC. It is suitable for graduate students and researchers and serves as a key reference for this specialized field. The book requires a solid background in probability and linear algebra.
Geared towards professionals, this book focuses on applying reinforcement learning in industrial settings. It covers the practical aspects of deploying RL solutions and offers insights into real-world use cases. It's particularly relevant for those looking to apply RL in their work.
Explores the application of reinforcement learning methods specifically within the domain of finance. It provides a Python-based introduction to relevant algorithms and their use in financial problems like algorithmic trading. It's a valuable resource for those with an interest in this specialized application area.
Discusses important RL algorithms and their convergence theory, including recent and advanced topics like deep reinforcement learning and distributional reinforcement learning. It delves into how modern algorithms work and is suitable for those seeking a deeper understanding of the theoretical guarantees.
Provides a comprehensive introduction to reinforcement learning, which type of machine learning that is used to solve sequential decision-making problems. It covers the basics of reinforcement learning, as well as more advanced topics such as deep reinforcement learning and multi-agent reinforcement learning.
Provides a concise and rigorous introduction to the algorithms of reinforcement learning. It is an excellent resource for those who want to deepen their understanding of the theoretical underpinnings of RL algorithms. While shorter than Sutton and Barto, it offers valuable insights and good supplementary read for a more mathematical perspective.
Practical guide for beginners wanting to start their journey in deep reinforcement learning using Python. It provides a hands-on approach to understanding and implementing DRL algorithms, making it suitable for those with some programming background looking to apply RL concepts.
Provides a comprehensive guide to decision-making under uncertainty, with a significant focus on reinforcement learning and Markov decision processes. It offers a solid foundation in the theoretical concepts underlying RL and is valuable for those interested in the broader context of sequential decision making.
This graduate-level textbook offers a clear and intuitive introduction to stochastic processes, including Markov Chains, with a focus on examples and applications alongside theoretical concepts. It is suitable for students looking to deepen their understanding with a less abstract approach. The book is well-regarded for its engaging writing style.
Connects the theory of finite Markov chains to important algorithmic applications, such as Markov Chain Monte Carlo methods. It is valuable for advanced undergraduate and graduate students in computer science and related quantitative fields. It helps solidify understanding by demonstrating the practical use of Markov chains in algorithms.
This recent publication aims to bridge the gap between the theory of Markov Chains and their practical implementation and applications across various disciplines. It is valuable for students and professionals who want to see the versatility of Markov Chains and learn how to apply them in different contexts. It good resource for exploring contemporary applications.
A rigorous graduate-level textbook on probability theory that includes a comprehensive treatment of Markov Chains from a theoretical perspective. It is essential for students pursuing advanced degrees in mathematics or statistics who require a deep, measure-theoretic understanding. is more valuable as a theoretical reference and prerequisite builder.
Provides a Bayesian perspective on Markov decision processes, which allows for the incorporation of uncertainty into the decision-making process. It is especially useful for readers who are interested in using MDPs to solve problems in which there is uncertainty about the state of the world.
Emphasizes the applications and computational aspects of stochastic processes, including Markov Chains, making it highly relevant for practitioners. It is suitable for graduate students and professionals interested in applying theoretical concepts to real-world problems. Published recently, it offers contemporary perspectives on applied stochastic processes.
Offers broad coverage of applied probability and stochastic processes, including significant material on Markov Chains and their applications in various fields such as engineering and operations research. It useful reference for professionals and graduate students interested in seeing how Markov chains are applied in practice. It provides a good breadth of applied topics.
This widely-used textbook covering the broad field of artificial intelligence, with dedicated chapters on reinforcement learning. While not exclusively an RL book, it provides essential context and covers foundational AI concepts that are relevant to understanding RL within the larger AI landscape. It's a valuable reference for a broad understanding of AI.
Uniquely combines the theory of stochastic processes, including Markov Chains, with practical implementation using the R programming language. It is ideal for students and professionals who want a hands-on approach to the subject. It useful reference for computational aspects and helps solidify understanding through practical examples.
Focuses on the modeling aspects of stochastic processes, demonstrating how they can be applied to various real-world phenomena that can be represented by Markov Chains. It is valuable for students and professionals interested in the practical utility of Markov Chains in building and analyzing models. It helps solidify understanding through applied examples.
Provides a detailed guide to stochastic simulation methods, including techniques like MCMC that are based on Markov Chains. It useful resource for graduate students and professionals who need to implement simulations in practice. While an older publication, the fundamental concepts and methods remain relevant for understanding simulation techniques.
This influential textbook in machine learning covers probabilistic models, including Hidden Markov Models, demonstrating the practical application of Markov Chains in areas like speech recognition and bioinformatics. It is valuable for graduate students and professionals interested in the intersection of Markov Chains and machine learning. It serves as a comprehensive reference for various ML techniques.
While covering broader machine learning concepts, this book includes fundamentals relevant to understanding the context of reinforcement learning. It good starting point for beginners in ML, providing foundational theory and ethical considerations. It can serve as preparatory reading before diving specifically into RL.
Provides an introduction to Markov decision processes (MDPs) and their applications in engineering. It covers the basics of MDPs, as well as more advanced topics such as reinforcement learning and planning.
Provides a comprehensive overview of Markov decision processes (MDPs) in discrete time. It covers the basics of MDPs, as well as more advanced topics such as reinforcement learning and planning.

Share

Help others find this course page by sharing it with your friends and followers:

Similar courses

Here are nine courses similar to Data Science Decisions in Time: Using Data Effectively.
Linear Algebra: Matrix Algebra, Determinants, &...
Most relevant
Advanced Neural Network Techniques
Most relevant
Engineering Genetic Circuits: Abstraction Methods
Most relevant
Text Generation with Markov Chains in Python
Most relevant
Bayesian Statistics: Time Series Analysis
Most relevant
Theory of Credit Risk Models
Most relevant
Introduction to Probability
Most relevant
Linear Algebra III: Determinants and Eigenvalues
Most relevant
Probability - The Science of Uncertainty and Data
Most relevant

Coursera Buyer Protection & Guarantee

Verified partner policies & purchase protections

Accredited University Partner
Free Trial & Audit

7-Day Free Trial & Free Audit

7-day free trial on monthly Coursera Plus subscriptions; free audit option available on most individual courses to explore video lectures and readings at no charge.

Return & Refund Policy

14-Day Refund Guarantee

14-day refund window on annual Coursera Plus subscriptions and single-course purchases prior to certificate issuance. Monthly subscriptions do not offer refunds once billed.

Buyer Protections

Verified Academic & Industry Partner

Accredited university and industry certifications (e.g. Stanford, Google, IBM, Yale), secure SSL payment processing, and verifiable digital certificate links.

Key Purchase Protections
14-day refund window on annual Coursera Plus and single courses
7-day free trial on monthly Coursera Plus subscriptions
Free course audit option for lectures and readings
Verified certificates from top universities & companies
OpenCourser Verified Partner Policy details last reviewed: 2026-10-22
Our mission

OpenCourser helps millions of learners each year. People visit us to learn workspace skills, ace their exams, and nurture their curiosity.

Our extensive catalog contains over 50,000 courses and twice as many books. Browse by search, by topic, or even by career interests. We'll match you to the right resources quickly.

Find this site helpful? Tell a friend about us.

Affiliate disclosure

We're supported by our community of learners. When you purchase or subscribe to courses and programs or purchase books, we may earn a commission from our partners.

Your purchases help us maintain our catalog and keep our servers humming without ads.

Thank you for supporting OpenCourser.

© 2016 - 2025 OpenCourser