A Complete Reinforcement Learning System (Capstone) from Coursera

In this final course, you will put together your knowledge from Courses 1, 2 and 3 to implement a complete RL solution to a problem. This capstone will let you see how each component---problem formulation, algorithm selection, parameter selection and representation design---fits together into a complete solution, and how to make appropriate choices when deploying RL in the real world. This project will require you to implement both the environment to stimulate your problem, and a control agent with Neural Network function approximation. In addition, you will conduct a scientific study of your learning system to develop your ability to assess the robustness of RL agents. To use RL in the real world, it is critical to (a) appropriately formalize the problem as an MDP, (b) select appropriate algorithms, (c ) identify what choices in your implementation will have large impacts on performance and (d) validate the expected behaviour of your algorithms. This capstone is valuable for anyone who is planning on using RL to solve real problems.

To be successful in this course, you will need to have completed Courses 1, 2, and 3 of this Specialization or the equivalent.

By the end of this course, you will be able to:

Complete an RL solution to a problem, starting from problem formulation, appropriate algorithm selection and implementation and empirical study into the effectiveness of the solution.

What's inside

Syllabus

Welcome to the Final Capstone Course!

Welcome to the final capstone course of the Reinforcement Learning Specialization!!

Milestone 1: Formalize Word Problem as MDP

This week you will read a description of a problem, and translate it into an MDP. You will complete skeleton code for this environment, to obtain a complete MDP for use in this capstone project.

Milestone 2: Choosing The Right Algorithm

This week you will select from three algorithms, to learn a policy for the environment. You will reflect on and discuss the appropriateness of each algorithm for this environment.

Milestone 3: Identify Key Performance Parameters

This week you will identify key parameters that affect the performance of your agent. The goal is to understand the space of options, to later enable you to choose which parameter you will investigate in-depth for your agent.

Milestone 4: Implement Your Agent

This week, you will implement your agent using Expected Sarsa or Q-learning with RMSProp and Neural Networks. To use NNs, you will have to use a more careful stepsize selection strategy, which is why you will use RMSProp. You will also verify the correctness of your agent.

Milestone 5: Submit Your Parameter Study!

This week you will identify a parameter to study, for your agent. Once you select the parameter to study, we will provide you with a range of values and specific values for other parameters. You will write a script to run your agent and environment on the set of parameters, to determine performance across these parameters. You will gain insight into the impact of parameters on agent performance. You will also get to visualize the agents that you learn. Your parameter study will consist of an array of values that we will check for correctness.

Good to know

Know what's good

, what to watch for

, and possible dealbreakers

Teaches RL problem formulation, algorithm selection, parameter selection and representation design from the perspective of completing an RL solution to a problem

Strengthens existing foundation for intermediate learners

Taught by instructors recognized for their work on the course topic

Takes a creative approach to an established topic, field, or subject

Develops deep expertise in a particular topic

Course explicitly requires learners to come in with extensive background knowledge first

Reviews summary

Capstone: build a reinforcement learning system

learners say this course is an excellent conclusion to a highly-rated machine learning specialization. According to students, the capstone project's step-by-step format and engaging assignments help learners put into practice what they have learned in the previous three courses. Students also remark that the course's easy-to-understand explanations, clear videos, and well-structured materials make it accessible to learners of all experience levels.

accessible and easy to understand

"This is the final chapter. It is one of the easiest and it was fun doing that lunar landing project."

"They mostly discuss the importance of real world experience and hyperparameter tuning in this class. The content it did have was solid and the instructors were great."

interesting and practical

"The Final project is Lunar Lander , applying what we learned in the previous courses in the specialisation."

"This course ties everything in the previous three courses together to simulate a reinforcement learning system for landing a lunar module on the Moon."

well-structured and logical

"The course material is clearly explained in logical steps to build intuition."

"After learning multiple complex but simple to understand aspects in the first courses, it all comes together in the later courses of the specialization."

Activities

Be better prepared before your course. Deepen your understanding during and after it. Supplement your coursework and achieve mastery of the topics covered in A Complete Reinforcement Learning System (Capstone) with these activities:

Read Sutton and Barto's Reinforcement Learning: An Introduction

Show steps

This book is a classic textbook on reinforcement learning and provides a comprehensive overview of the field.

View Reinforcement Learning, second edition: An... on Amazon

Show steps

Read through the book carefully, making sure to understand each chapter.
Take notes and highlight important passages.
Complete the exercises at the end of each chapter.

Show all one activities

Career center

Learners who complete A Complete Reinforcement Learning System (Capstone) will develop knowledge and skills that may be useful to these careers:

Artificial Intelligence Engineer

Artificial Intelligence Engineers build and maintain AI software systems and implement machine learning algorithms. They work on the research and development of AI technologies such as natural language processing, computer vision, and reinforcement learning. This course provides a solid foundation in reinforcement learning, which is a key component of AI systems. By completing this course, you will gain the knowledge and skills necessary to build and deploy RL solutions to real-world problems, making you a valuable asset to any organization working in the field of AI.

See salaries and explore the career path for Artificial Intelligence Engineer

Machine Learning Engineer

Machine Learning Engineers design, develop, and deploy machine learning models to solve real-world problems. They work on a variety of tasks, such as data preprocessing, feature engineering, model selection, and model evaluation. This course provides a comprehensive overview of reinforcement learning, a powerful machine learning technique that enables agents to learn optimal behavior in complex environments. By completing this course, you will gain the skills and knowledge necessary to develop and deploy RL solutions, making you a sought-after candidate in the field of machine learning.

See salaries and explore the career path for Machine Learning Engineer

Data Scientist

Data Scientists use data to solve business problems. They work on a variety of tasks, such as data analysis, data visualization, and predictive modeling. This course provides a strong foundation in reinforcement learning, which is a powerful technique for solving complex decision-making problems. By completing this course, you will gain the skills and knowledge necessary to develop and deploy RL solutions to real-world problems, making you a valuable asset to any organization that uses data to make decisions.

See salaries and explore the career path for Data Scientist

Software Engineer

Software Engineers design, develop, and maintain software systems. They work on a variety of tasks, such as requirements gathering, software design, coding, and testing. This course provides a comprehensive overview of reinforcement learning, a powerful technique for solving complex decision-making problems. By completing this course, you will gain the skills and knowledge necessary to develop and deploy RL solutions to real-world problems, making you a valuable asset to any organization that develops software.

See salaries and explore the career path for Software Engineer

Quantitative Analyst

Quantitative Analysts use mathematical and statistical models to solve financial problems. They work on a variety of tasks, such as risk management, portfolio optimization, and trading strategies. This course provides a strong foundation in reinforcement learning, which is a powerful technique for solving complex decision-making problems under uncertainty. By completing this course, you will gain the skills and knowledge necessary to develop and deploy RL solutions to real-world financial problems, making you a valuable asset to any financial institution.

See salaries and explore the career path for Quantitative Analyst

Operations Research Analyst

Operations Research Analysts use mathematical and statistical models to solve business problems. They work on a variety of tasks, such as supply chain management, logistics, and scheduling. This course provides a comprehensive overview of reinforcement learning, a powerful technique for solving complex decision-making problems under uncertainty. By completing this course, you will gain the skills and knowledge necessary to develop and deploy RL solutions to real-world business problems, making you a valuable asset to any organization that seeks to improve its operations.

See salaries and explore the career path for Operations Research Analyst

Computer Scientist

Computer Scientists research and develop new computing technologies. They work on a variety of topics, such as artificial intelligence, machine learning, and computer vision. This course provides a strong foundation in reinforcement learning, a powerful technique for solving complex decision-making problems. By completing this course, you will gain the skills and knowledge necessary to develop and deploy RL solutions to real-world problems, making you a valuable asset to any organization that develops new computing technologies.

See salaries and explore the career path for Computer Scientist

Data Analyst

Data Analysts use data to solve business problems. They work on a variety of tasks, such as data analysis, data visualization, and data mining. This course provides a strong foundation in reinforcement learning, which is a powerful technique for solving complex decision-making problems. By completing this course, you will gain the skills and knowledge necessary to develop and deploy RL solutions to real-world business problems, making you a valuable asset to any organization that uses data to make decisions.

See salaries and explore the career path for Data Analyst

Business Analyst

Business Analysts work with businesses to identify and solve problems. They use a variety of tools and techniques to analyze business processes, identify inefficiencies, and develop solutions. This course provides a comprehensive overview of reinforcement learning, a powerful technique for solving complex decision-making problems. By completing this course, you will gain the skills and knowledge necessary to develop and deploy RL solutions to real-world business problems, making you a valuable asset to any organization that seeks to improve its operations.

See salaries and explore the career path for Business Analyst

Statistician

Statisticians collect, analyze, and interpret data. They work on a variety of topics, such as data analysis, data visualization, and statistical modeling. This course provides a strong foundation in reinforcement learning, a powerful technique for solving complex decision-making problems under uncertainty. By completing this course, you will gain the skills and knowledge necessary to develop and deploy RL solutions to real-world problems, making you a valuable asset to any organization that uses data to make decisions.

See salaries and explore the career path for Statistician

Financial Analyst

Financial Analysts use financial data to make investment decisions. They work on a variety of tasks, such as analyzing financial statements, evaluating investment opportunities, and developing investment strategies. This course provides a strong foundation in reinforcement learning, a powerful technique for solving complex decision-making problems under uncertainty. By completing this course, you will gain the skills and knowledge necessary to develop and deploy RL solutions to real-world financial problems, making you a valuable asset to any financial institution.

See salaries and explore the career path for Financial Analyst

Actuary

Actuaries use mathematical and statistical models to assess risk. They work on a variety of tasks, such as pricing insurance policies, developing risk management strategies, and forecasting financial outcomes. This course provides a strong foundation in reinforcement learning, a powerful technique for solving complex decision-making problems under uncertainty. By completing this course, you will gain the skills and knowledge necessary to develop and deploy RL solutions to real-world risk management problems, making you a valuable asset to any organization that seeks to manage risk.

See salaries and explore the career path for Actuary

Economist

Economists study the production, distribution, and consumption of goods and services. They work on a variety of topics, such as economic growth, unemployment, and inflation. This course provides a strong foundation in reinforcement learning, a powerful technique for solving complex decision-making problems. By completing this course, you will gain the skills and knowledge necessary to develop and deploy RL solutions to real-world economic problems, making you a valuable asset to any organization that seeks to improve economic outcomes.

See salaries and explore the career path for Economist

Policy Analyst

Policy Analysts develop and evaluate public policies. They work on a variety of topics, such as education, healthcare, and environmental protection. This course provides a comprehensive overview of reinforcement learning, a powerful technique for solving complex decision-making problems. By completing this course, you will gain the skills and knowledge necessary to develop and deploy RL solutions to real-world policy problems, making you a valuable asset to any organization that seeks to improve public policy.

See salaries and explore the career path for Policy Analyst

Consultant

Consultants provide advice and guidance to businesses and organizations. They work on a variety of topics, such as strategy, operations, and technology. This course provides a comprehensive overview of reinforcement learning, a powerful technique for solving complex decision-making problems. By completing this course, you will gain the skills and knowledge necessary to develop and deploy RL solutions to real-world business problems, making you a valuable asset to any consulting firm.

See salaries and explore the career path for Consultant