We may earn an affiliate commission when you visit our partners.

ML Model Scoring and Monitoring

Giacomo Vianello, Ulrika Jägare, Justin Clifford Smith, PhD, Bradford Tuckfield, and Joshua Bernhard

This course will help students automate the devops processes required to score and re-deploy ML models. Students will automate model training and deployment. They will set up regular scoring processes to be performed after model deployment, and also learn to reason carefully about model drift, and whether models need to be retrained and re-deployed. Students will learn to diagnose operational issues with models, including data integrity and stability problems, timing problems, and dependency issues. Finally, students will learn to set up automated reporting with API’s.

Enroll now

What's inside

Syllabus

This lesson will talk about goals for the course, when to use ML model scoring and monitoring, stakeholders, the history of the field, and the tools and dependencies you need to be aware of.

This lesson will talk about model training and deployment. We’ll focus on automating the training and deployment process and making sure that the trained, deployed models are ready to be monitored.

This lesson will discuss model scoring and model drift, an important part of the continuous monitoring that makes sure your deployed model remains as accurate as possible.

There are problems that can come up in deployed projects. So this lesson will talk about diagnosing and fixing operational problems, a crucial part of the post-deployment machine learning process.

This lesson will discuss model reporting and monitoring with APIs which can be used as an automatic interface with your ML project.

The final project for this course will be a dynamic risk assessment system in which you will build and monitor an ML model to predict attrition risk.

Good to know

Know what's good

, what to watch for

, and possible dealbreakers

Explores the fundamentals of ML model automation, which are highly relevant in industry

Develops the skills for building systems that deploy ML models, which are valuable in a variety of work contexts

Taught by industry experts, who are recognized for their work in deploying ML models

Equips learners with knowledge of diagnostic and debugging tools in model deployment, which are critical for managing ML model performance

Requires learners to have a strong foundation in ML model building, deployment, and evaluation, which may limit accessibility for learners who are new to these concepts

Save this course

Save ML Model Scoring and Monitoring to your list so you can find it easily later:

Save

Activities

Coming soon We're preparing activities for ML Model Scoring and Monitoring. These are activities you can do either before, during, or after a course.

Career center

Learners who complete ML Model Scoring and Monitoring will develop knowledge and skills that may be useful to these careers:

Machine Learning Researcher

This course may be of interest to Machine Learning Researchers who want to gain practical experience in ML model scoring and monitoring. You will learn to set up regular scoring processes and diagnose operational issues with deployed ML models. Furthermore, you will become familiar with the tools and techniques used in this field. This knowledge can enhance your skills as a Machine Learning Researcher and provide you with a competitive advantage in the job market.

See salaries and explore the career path for Machine Learning Researcher

Data Scientist

In the field of Data Science, professionals must be familiar with the tools and techniques of DevOps processes for ML model scoring and monitoring. The course will help you become familiar with these DevOps processes. Specifically, you will learn to reason about model drift and diagnose operational issues with deployed ML models. This will help you ensure your deployed models remain accurate and stable throughout their lifecycle.

See salaries and explore the career path for Data Scientist

Statistician

This course may prove useful to Statisticians who want to specialize in Machine Learning. You will learn to set up regular scoring processes and diagnose operational issues with deployed ML models. Furthermore, you will gain experience in reasoning about model drift. This knowledge can enhance your skills as a Statistician and open up new opportunities in the field of Machine Learning.

See salaries and explore the career path for Statistician

Data Engineer

This course may prove useful to Data Engineers who want to gain proficiency in Machine Learning. You will learn to automate ML model training and deployment, as well as monitor deployed models for performance issues. This knowledge can enhance your skills as a Data Engineer and open up new opportunities in the field of Machine Learning.

See salaries and explore the career path for Data Engineer

Data Analyst

A Data Analyst may benefit from taking this course to gain a foundation in Machine Learning. You will learn to automate ML model training and deployment, as well as monitor deployed models for performance issues. This knowledge can enhance your skills as a Data Analyst and open up new opportunities in the realm of Machine Learning.

See salaries and explore the career path for Data Analyst

Quantitative Analyst

This course may be of interest to Quantitative Analysts interested in gaining proficiency in Machine Learning. The course will teach you the concepts of model training and deployment, as well as how to set up regular scoring processes. You will learn to diagnose operational issues with deployed ML models and become familiar with the tools and techniques commonly used in ML model scoring and monitoring.

See salaries and explore the career path for Quantitative Analyst

Cloud Architect

This course may be of interest to Cloud Architects who want to gain proficiency in Machine Learning. You will learn to automate ML model training and deployment, as well as monitor deployed models for performance issues. This knowledge can enhance your skills as a Cloud Architect and open up new opportunities in the field of Machine Learning.

See salaries and explore the career path for Cloud Architect

Data Architect

This course may prove useful to Data Architects who want to gain proficiency in Machine Learning. You will learn to automate ML model training and deployment, as well as monitor deployed models for performance issues. This knowledge can enhance your skills as a Data Architect and open up new opportunities in the field of Machine Learning.

See salaries and explore the career path for Data Architect

Machine Learning Engineer

A Machine Learning Engineer may utilize automated model training and deployment, as covered in this course. You will diagnose operational issues through reasoning of model drift and thus make informed decisions on whether to retrain and redeploy deployed ML models. This course may help you build a foundation to succeed in this role.

See salaries and explore the career path for Machine Learning Engineer

DevOps Engineer

A DevOps Engineer may find this course helpful in gaining a foundation in Machine Learning. You will become familiar with the tools and techniques used in ML model scoring and monitoring and learn to diagnose operational issues with deployed ML models. This knowledge can enhance your skills as a DevOps Engineer and open up new opportunities in the field of Machine Learning.

See salaries and explore the career path for DevOps Engineer

Operations Research Analyst

An Operations Research Analyst may find this course helpful in gaining a foundation in Machine Learning. You will become familiar with the tools and techniques used in ML model scoring and monitoring and learn to diagnose operational issues with deployed ML models. This knowledge can enhance your skills as an Operations Research Analyst and open up new opportunities in the field of Machine Learning.

See salaries and explore the career path for Operations Research Analyst

Business Analyst

This course may prove useful for Business Analysts who want to gain proficiency in Machine Learning. You will learn to understand the goals and stakeholders involved in ML model scoring and monitoring. Furthermore, you will become familiar with the tools and techniques used in this field. This knowledge can enhance your value as a Business Analyst and provide you with a competitive edge in the job market.

See salaries and explore the career path for Business Analyst

Product Manager

A Product Manager may find this course helpful in gaining a foundational understanding of Machine Learning. You will become familiar with the tools and techniques used in ML model scoring and monitoring and learn to reason about model drift. This knowledge can enable you to make more informed decisions about ML-related products and services.

See salaries and explore the career path for Product Manager

Software Developer

A Software Developer may find this course helpful in gaining a foundation in Machine Learning. You will become familiar with the tools and techniques used in ML model scoring and monitoring and learn to reason about model drift. This knowledge can enhance your skills as a Software Developer and open up new opportunities in the field of Machine Learning.

See salaries and explore the career path for Software Developer

Software Engineer

This course may prove helpful to a Software Engineer who wants to specialize in Machine Learning. You will learn to automate ML model training and deployment. Furthermore, you will gain experience in diagnosing operational issues with deployed ML models. These skills may increase your value as a Software Engineer and open up new opportunities in Machine Learning.

See salaries and explore the career path for Software Engineer

Reading list

We've selected ten books that we think will supplement your learning. Use these to develop background knowledge, enrich your coursework, and gain a deeper understanding of the topics covered in ML Model Scoring and Monitoring.

Introducing MLOps