Measuring Total Data Quality from Coursera

By the end of this second course in the Total Data Quality Specialization, learners will be able to:

1. Learn various metrics for evaluating Total Data Quality (TDQ) at each stage of the TDQ framework.

2. Create a quality concept map that tracks relevant aspects of TDQ from a particular application or data source.

3. Think through relative trade-offs between quality aspects, relative costs and practical constraints imposed by a particular project or study.

4. Identify relevant software and related tools for computing the various metrics.

5. Understand metrics that can be computed for both designed and found/organic data.

6. Apply the metrics to real data and interpret their resulting values from a TDQ perspective.

This specialization as a whole aims to explore the Total Data Quality framework in depth and provide learners with more information about the detailed evaluation of total data quality that needs to happen prior to data analysis. The goal is for learners to incorporate evaluations of data quality into their process as a critical component for all projects. We sincerely hope to disseminate knowledge about total data quality to all learners, such as data scientists and quantitative analysts, who have not had sufficient training in the initial steps of the data science process that focus on data collection and evaluation of data quality. We feel that extensive knowledge of data science techniques and statistical analysis procedures will not help a quantitative research study if the data collected/gathered are not of sufficiently high quality.

This specialization will focus on the essential first steps in any type of scientific investigation using data: either generating or gathering data, understanding where the data come from, evaluating the quality of the data, and taking steps to maximize the quality of the data prior to performing any kind of statistical analysis or applying data science techniques to answer research questions. Given this focus, there will be little material on the analysis of data, which is covered in myriad existing Coursera specializations. The primary focus of this specialization will be on understanding and maximizing data quality prior to analysis.

What's inside

Syllabus

Introduction and Measuring Validity and Data Origin Quality

Welcome to Measuring Total Data Quality! This is the second course in the Total Data Quality Specialization. After reviewing the Course 2 syllabus and completing the course pre-survey, you’ll learn how to measure validity for designed and gathered data through a series of video lectures, examples, and readings. You’ll then take a short quiz on interpreting validity metrics. Then, you’ll complete a module on data origin, where you’ll learn about measuring data origin quality for designed and gathered data in a series of video lectures and case studies. Week 1 will conclude with a quiz on interpreting data origin quality metrics.

Measuring Processing and Data Access Quality

Welcome to Week 2 of Measuring Total Data Quality! We’ll begin the week by discussing how to measure processing data quality for designed and gathered data. We’ll include examples of measuring process data quality for each form of data and conclude the module with a quiz on interpreting processing metrics. In the second half of Week 2, we’ll discuss measuring data access quality for designed and gathered data through video lectures, an example, and a case study, and conclude the week with a quiz on interpreting access metrics.

Measuring Data Source Quality and Data Missingness

This week, we’ll learn how to measure data source quality and data missingness. We’ll begin Week 3 with a video lecture on measuring data source quality for designed data. Then, we’ll work through an example of computing data source metrics with real data and code. We’ll then learn how to measure data source quality for gathered data and see an example of computer data source quality metrics with real data and code. You’ll then take a short quiz on interpreting data source quality metrics and move on to the Data Missingness unit. We’ll learn how to measure threats to data source quality for designed and gathered data and work through examples for each form of data. Week 3 will conclude with a quiz on interpreting data missingness metrics.

Measuring the Quality of Data Analysis

We’ll be wrapping up Measuring Total Data Quality this week by learning how to measure the quality of data analysis. We’ll learn how to measure the quality of data analysis for designed and gathered data and work through examples of each type of data. We recommend that you complete two readings before you complete the lecture on measuring the quality of analysis for gathered data. We will conclude the week with a quiz on examining quality metrics and interpreting output, as well as references for the Measuring Total Data Quality course and a course post-survey.

Good to know

Know what's good

, what to watch for

, and possible dealbreakers

Develops skills in quantifying overall data quality critical for all data science and quantitative analyst projects

This course strengthens an existing foundational understanding of data collection and data science

Provides a comprehensive examination of tools and methods for evaluating data quality

Builds a strong foundation for beginners in the field of data quality

Teaches techniques for maximizing data quality before conducting analysis

Taught by Brady West, James Wagner, Jinseok Kim, and Trent Buskirk

Activities

Be better prepared before your course. Deepen your understanding during and after it. Supplement your coursework and achieve mastery of the topics covered in Measuring Total Data Quality with these activities:

Complete the Data Quality Online Primer

Show steps

Refreshes your knowledge of data quality concepts before starting the course, filling any gaps in understanding.

Browse courses on Data Quality

Show steps

Read Data Quality and Data Quality Dimensions
Enroll in a free Data Quality Online Primer course on Coursera

Intro to Data Quality Concepts

Show steps

This will provide the foundational knowledge necessary to execute the techniques discussed in the course.

Browse courses on Data Quality

Show steps

Review data quality definitions and concepts.
Identify the different dimensions of data quality.
Discuss the importance of data quality in data analysis.

Review design and data quality metrics

Show steps

Brush up on data quality metrics before starting the course to refresh your mind for greater clarity and understanding during the classes.

Show steps

Read the TDQ Framework Reference Guide
Complete the TDQ Self-Assessment Tool

Eight other activities

Expand to see all activities and additional details

Show all 11 activities

Review Data Structures and Algorithms

Show steps

This activity will help you refresh your fundamental skills in data structures and algorithms to better prepare yourself for the content of this course.

Browse courses on Data Structures

Show steps

Review common data structure implementations such as arrays, linked lists, stacks, and queues.
Practice basic algorithms for searching, sorting, and traversal.
Complete coding exercises on platforms like LeetCode or HackerRank.

Create a Data Quality Plan

Show steps

Develop a plan to ensure data quality throughout the course projects, improving the reliability and consistency of your data analysis.

Show steps

Define data quality objectives
Identify data sources and collection methods
Establish data quality standards and metrics
Consider data management and storage strategies

Data Quality Assessment using OpenRefine

Show steps

Build familiarity with tools and techniques for data quality assessment.

Browse courses on Data Quality Assessment

Show steps

Find and install OpenRefine.
Load a dataset into OpenRefine.
Explore the data and identify potential quality issues.
Use OpenRefine's built-in tools to clean and transform the data.
Export the cleaned data to a new file.

Practice Data Quality Calculations

Show steps

Sharpen your skills in calculating and interpreting data quality metrics, improving your ability to assess data quality.

Show steps

Calculate Data Accuracy, Completeness, Consistency, and Validity
Use data quality tools to automate calculations

Data Quality Metrics Calculation

Show steps

Increase proficiency in calculating and interpreting data quality metrics.

Show steps

Learn the formulas for calculating common data quality metrics, such as completeness, accuracy, and consistency.
Practice calculating these metrics using real-world datasets.
Interpret the results of your calculations and identify potential data quality issues.

Data Quality Best Practices Guide

Show steps

Solidify understanding of data quality principles by creating a comprehensive resource.

Show steps

Research and gather information on data quality best practices.
Organize the information into a logical structure.
Write clear and concise instructions on how to implement these best practices.
Share the guide with others.

Kaggle Data Quality Challenge

Show steps

Gain practical experience in applying data quality techniques and showcase your skills.

Browse courses on Kaggle

Show steps

Find and register for a Kaggle competition focused on data quality.
Download the competition dataset and explore it.
Develop and implement a data quality pipeline to clean and transform the data.
Submit your results to the competition and compare them to other participants.

Data Quality Report for a Real-World Dataset

Show steps

Demonstrate mastery of data quality concepts through a comprehensive analysis.

Show steps

Choose a real-world dataset that is relevant to your interests or work.
Assess the quality of the dataset using the techniques covered in the course.
Write a report that summarizes your findings and provides recommendations for improving the data quality.

Career center

Learners who complete Measuring Total Data Quality will develop knowledge and skills that may be useful to these careers:

Data Quality Analyst

A Data Quality Analyst is responsible for ensuring that data is accurate, reliable, and consistent. They develop and implement data quality standards and procedures, and monitor data quality to identify and resolve issues. This course can be useful for aspiring Data Quality Analysts as it will provide them with the skills and knowledge needed to effectively evaluate and improve data quality.

See salaries and explore the career path for Data Quality Analyst

Data Analyst

A Data Analyst is responsible for transforming raw data into actionable insights that can help organizations make informed decisions. They help identify and remove errors in datasets. They gather data and ensure that data from different sources is valid and accurate. Measuring Total Data Quality is an excellent course for aspiring Data Analysts as it will provide them with the essential understanding of different metrics for evaluating Total Data Quality, and the skills to identify and address data quality issues.

See salaries and explore the career path for Data Analyst

Data Scientist

Data Scientists develop and implement mathematical and statistical models to extract insights from data. They are responsible for ensuring that the data they use is accurate and reliable. This course is an excellent choice for aspiring Data Scientists as it will provide them with the skills and knowledge needed to assess the quality of data and identify and address data quality issues.

See salaries and explore the career path for Data Scientist

Quality Assurance Analyst

A Quality Assurance Analyst is responsible for testing and verifying the quality of products and services. They work with data analysts and other stakeholders to identify and address data quality issues. This course can be useful for aspiring Quality Assurance Analysts as it will provide them with the skills and knowledge needed to understand and evaluate data quality.

See salaries and explore the career path for Quality Assurance Analyst

Data Engineer

A Data Engineer designs and builds systems to store, manage, and process data. They are responsible for ensuring that data is accurate, reliable, and accessible to data analysts and other users. The Measuring Total Data Quality course can be useful for aspiring Data Engineers as it will help them understand the importance of data quality and the different methods for evaluating and improving data quality.

See salaries and explore the career path for Data Engineer

Information Systems Manager

An Information Systems Manager is responsible for planning, implementing, and maintaining information systems. They are responsible for ensuring that data is accurate, reliable, and accessible to users. The Measuring Total Data Quality course can be useful for aspiring Information Systems Managers as it will provide them with the skills and knowledge needed to understand and evaluate data quality.

See salaries and explore the career path for Information Systems Manager

Operations Research Analyst

An Operations Research Analyst uses data to improve the efficiency and effectiveness of organizations. They work with data analysts and other stakeholders to identify and address data quality issues. This course can be useful for aspiring Operations Research Analysts as it will provide them with the skills and knowledge needed to understand and evaluate data quality.

See salaries and explore the career path for Operations Research Analyst

Market Researcher

A Market Researcher collects and analyzes data to understand customer needs and preferences. They work with data analysts and other stakeholders to identify and address data quality issues. This course can be useful for aspiring Market Researchers as it will provide them with the skills and knowledge needed to understand and evaluate data quality.

See salaries and explore the career path for Market Researcher

Systems Analyst

A Systems Analyst designs and develops computer systems. They work with data analysts and other stakeholders to identify and address data quality issues. This course can be useful for aspiring Systems Analysts as it will provide them with the skills and knowledge needed to understand and evaluate data quality.

See salaries and explore the career path for Systems Analyst

Information Architect

An Information Architect designs and organizes information systems to ensure that they are easy to use and meet the needs of users. They work closely with data analysts and other stakeholders to identify and address data quality issues. This course can be useful for aspiring Information Architects as it will provide them with the skills and knowledge needed to understand and evaluate data quality.

See salaries and explore the career path for Information Architect

Knowledge Manager

A Knowledge Manager is responsible for creating and managing knowledge within an organization. They work with data analysts and other stakeholders to identify and address data quality issues. This course can be useful for aspiring Knowledge Managers as it will provide them with the skills and knowledge needed to understand and evaluate data quality.

See salaries and explore the career path for Knowledge Manager

Software Engineer

A Software Engineer designs, develops, and maintains software applications. They work with data analysts and other stakeholders to identify and address data quality issues. This course can be useful for aspiring Software Engineers as it will provide them with the skills and knowledge needed to understand and evaluate data quality.

See salaries and explore the career path for Software Engineer

Risk Analyst

A Risk Analyst identifies and assesses risks to an organization. They work with data analysts and other stakeholders to identify and address data quality issues. This course can be useful for aspiring Risk Analysts as it will provide them with the skills and knowledge needed to understand and evaluate data quality.

See salaries and explore the career path for Risk Analyst

Product Manager

A Product Manager is responsible for the development and management of products. They work with data analysts and other stakeholders to identify and address data quality issues. This course can be useful for aspiring Product Managers as it will provide them with the skills and knowledge needed to understand and evaluate data quality.

See salaries and explore the career path for Product Manager

Library Scientist

A Library Scientist is responsible for organizing and managing information resources. They work with data analysts and other stakeholders to identify and address data quality issues. This course can be useful for aspiring Library Scientists as it will provide them with the skills and knowledge needed to understand and evaluate data quality.

See salaries and explore the career path for Library Scientist