We may earn an affiliate commission when you visit our partners.
Course image
Ming-Long Lam and Jawahar Panchal

This course introduces the necessary concepts and common techniques for analyzing data. The primary emphasis is on the process of data analysis, including data preparation, descriptive analytics, model training, and result interpretation. The process starts with removing distractions and anomalies, followed by discovering insights, formulating propositions, validating evidence, and finally building professional-grade solutions. Following the process properly, regularly, and transparently brings credibility and increases the impact of the results.

Read more

This course introduces the necessary concepts and common techniques for analyzing data. The primary emphasis is on the process of data analysis, including data preparation, descriptive analytics, model training, and result interpretation. The process starts with removing distractions and anomalies, followed by discovering insights, formulating propositions, validating evidence, and finally building professional-grade solutions. Following the process properly, regularly, and transparently brings credibility and increases the impact of the results.

This course will cover topics including Exploratory Data Analysis, Feature Screening, Segmentation, Association Rules, Nearest Neighbors, Clustering, Decision Tree, Linear Regression, Logistic Regression, and Performance Evaluation. Besides, this course will review statistical theory, matrix algebra, and computational techniques as necessary.

This course prepares students ready for and capable of the data preparation and analysis process. Besides developing Python codes for carrying out the process, students will learn to tune the software tools for the most efficient implementation and optimal performance. At the end of this course, students will have built their inventory of data analysis codes and their confidence in advocating their propositions to the business stakeholders.

Required Textbook: This course does not mandate any textbooks because the lecture notes are self-contained.

Optional Materials: A Practitioner's Guide to Machine Learning (abbreviated PGML for Reading)

Software Requirements: Python version 3.11 or above with the latest compatible versions of NumPy, SciPy, Pandas, Scikit-learn, and Statsmodels libraries.

To succeed in this course, learners should possess a basic knowledge of linear algebra and statistics, basic set theory and probability theory, and have basic Python and SQL skills. A few courses that can help equip you with the database knowledge needed for this course are: Introduction to Relational Databases, Relational Database Design, and Relational Database Implementation and Applications.

Enroll now

What's inside

Syllabus

Module 1: Process of Preparing and Analyzing Data
Welcome to Data Preparation and Analysis! Module 1 guides students through the art of crafting informative and visually appealing histograms, a fundamental aspect of data visualization. Students will learn techniques for measuring the location and scale of data, understanding the origins and impacts of noise and missing values in datasets. This module also introduces the CRISP-DM Process, a structured approach to data mining, along with Gartner's Analytics Ascendancy Model for advanced data analysis. Additionally, students will explore the distinction between raw data and processed information, a key concept for effective data interpretation and decision-making.
Read more
Module 2: Measure and Visualize Correlation
Module 2 delves into the intricacies of statistical analysis, beginning with a thorough understanding of the p-value concept and its significance as a Type I Error indicator. Students will learn to apply statistical tests in Python to identify significantly correlated features, exploring various correlation metrics tailored for categorical, mixed-type, and continuous features. This module emphasizes practical application, equipping students with the skills to calculate and interpret these metrics using Python, thereby enhancing their ability to conduct sophisticated data analysis and draw meaningful conclusions from complex datasets.
Module 3: Market Basket Analysis
Module 3 offers a deep dive into the world of Association Rules, teaching students how to improvise these rules for identifying valuable feature combinations that generate specific label values. Learners will master setting appropriate thresholds for Support and Confidence and gain a comprehensive understanding of the Apriori Algorithm and the significance of Frequent Itemsets within it. This module covers the calculation of common metrics for Association Rules, familiarizing students with the relevant terminology. Additionally, learners will explore the practical application of Association Rules in Market Basket Analysis, including strategies for cross-selling, up-selling, and product bundling, equipping them with valuable skills for advanced data-driven decision making in business contexts.
Module 4: Partitioning, Segmenting, and Clustering of Observations
In Module 4, students will learn how to describe and interpret profiles of clusters, gaining proficiency in deploying the K-Means and K-Modes clustering algorithms. They will explore the application of Recency, Frequency, and Monetary (RFM) Analysis to identify the most valuable customers in retail business settings. The module also covers the technique of Simple Random Sampling with the option of incorporating stratification variables, enhancing the precision of data analysis. Furthermore, it emphasizes the importance of objectively validating models using a testing partition, ensuring the reliability and effectiveness of the analytical models in real-world scenarios.
Module 5: Linear Regression
This module delves into feature importance analysis in machine learning, covering Shapley Values, feature selection methods, statistical evaluation, feature interaction, aliasing, and the Least Squares Algorithm. Students will be able to master these concepts to build robust and interpretable models.
Module 6: Binary Logistic Regression
In Module 6, students will master the art of feature selection in machine learning by exploring the Forward and Backward Selection Method, the All-Possible Subsets Method, and the concept of complete and quasi-complete separation. Students will also discover association rules for identifying separations, interpret model parameters and predicted probabilities, and delve into the concepts of maximum likelihood estimation, odds, and odds ratios.
Module 7: Decision Trees - The CART Algorithm
Module 7 will equip students wth the ability to harness the power of tree-based models to uncover hidden patterns in your data. Students will be able to describe clusters effectively, intelligently set algorithm parameters, construct business rules from tree results, and utilize variance metrics, entropy values, and Gini indices for optimal tree construction.
Module 8: Evaluating the Performance of Models
Module 8 delves into the realm of evaluation metrics for machine learning models. Students will master the concepts of precision and recall curves, lift curves, and receiver operating characteristics (ROC) curves. Additionally, students will obtain the ability to discover methods for calculating probability thresholds using Kolmogorov-Smirnov statistics and F1 scores. They will be able to explore metrics like misclassification rate, area under the curve (AUC), and root mean squared error (RMSE), along with techniques for computing RMSE and detecting severely misfitted observations using model-specific residuals.
Summative Course Assessment
This module contains the summative course assessment that has been designed to evaluate your understanding of the course material and assess your ability to apply the knowledge you have acquired throughout the course. Be sure to review the course material thoroughly before taking the assessment.

Good to know

Know what's good
, what to watch for
, and possible dealbreakers
Taught by recognized experts in the field
Develops core data analysis skills for data preparation, descriptive analytics, model training, and result interpretation
Emphasizes practical applications of data analysis techniques
Covers a comprehensive range of data analysis topics, including EDA, feature screening, segmentation, association rules, nearest neighbors, clustering, decision trees, linear regression, logistic regression, and performance evaluation
Provides clear and detailed lecture notes that are self-contained
Requires a strong foundation in linear algebra, statistics, and Python programming

Save this course

Save Data Preparation and Analysis to your list so you can find it easily later:
Save

Activities

Be better prepared before your course. Deepen your understanding during and after it. Supplement your coursework and achieve mastery of the topics covered in Data Preparation and Analysis with these activities:
Review Intermediate Python and Statistics
Recall the relevant materials on Python and Statistics that you already know to minimize the learning curve when working on the harder concepts in this course.
Browse courses on Python
Show steps
  • Review the Python syntax and semantics.
  • Explore various statistical concepts like mean, median, and variance.
Review of Introduction to Applied Linear Algebra by Gilbert Strang
Supplement the matrix algebra concepts covered in this course with insights from a renowned author.
Show steps
  • Review the concepts of matrices, vectors, and linear transformations.
  • Solve systems of linear equations and matrix equations.
Reinforce Your Understanding of CRISP-DM and Gartner's Analytics Ascendancy Model
Become more proficient in the CRISP-DM process and Gartner's Analytics Ascendancy Model to enhance your data analysis and mining skills.
Browse courses on CRISP-DM
Show steps
  • Review the official CRISP-DM website and documentation.
  • Explore Gartner's website for resources on the Analytics Ascendancy Model.
  • Complete online tutorials or courses on CRISP-DM and the Analytics Ascendancy Model.
Eight other activities
Expand to see all activities and additional details
Show all 11 activities
Practice Exploratory Data Analysis Using Python
Build your proficiency in EDA techniques by solving hands-on exercises and assignments.
Browse courses on Exploratory Data Analysis
Show steps
  • Load and explore a dataset using Python.
  • Plot histograms and scatterplots to visualize data.
Participate in Discussion Forums
Engage with other students by sharing insights and asking questions on course topics.
Show steps
  • Join discussion forums.
  • Post questions and comments to initiate discussions.
Create a Data Analysis Project
Develop a data analysis project from start to finish to practice the process of data preparation and analysis.
Browse courses on Data Analysis
Show steps
  • Choose a dataset
  • Clean and prepare the data
  • Explore the data
  • Build a model
  • Evaluate the model
Guided Tutorials on Machine Learning with Python
Expand your understanding of different concepts and algorithms used in Machine Learning with the help of structured tutorials.
Browse courses on Machine Learning
Show steps
  • Explore tutorials on supervised and unsupervised learning algorithms.
  • Implement and practice these algorithms using Python code.
Strengthen Your Correlation Metrics Calculation Skills
Improve your ability to calculate and interpret correlation metrics using Python, enhancing your data analysis capabilities.
Browse courses on Statistical Analysis
Show steps
  • Review the theory behind correlation metrics.
  • Practice calculating correlation metrics using Python code.
  • Apply correlation metrics to real-world datasets and interpret the results.
Develop a Blog Post Explaining Feature Importance in Machine Learning
Enhance your understanding of feature importance techniques by explaining them to others.
Browse courses on Feature Importance
Show steps
  • Summarize the concept of feature importance.
  • Describe various methods for calculating feature importance.
  • Illustrate with examples and provide code snippets.
Participate in Kaggle Competitions
Test and showcase your data analysis skills by participating in real-world competitions.
Browse courses on Machine Learning
Show steps
  • Join Kaggle and explore available competitions.
  • Select a competition that aligns with your interests and skill level.
  • Build and submit your models.
Build a Data Analysis Dashboard Using Dash
Solidify your understanding of data visualization by creating an interactive dashboard.
Browse courses on Data Visualization
Show steps
  • Connect to a data source and extract relevant data.
  • Design and develop interactive visualizations using Dash.
  • Deploy the dashboard and share it with others.

Career center

Learners who complete Data Preparation and Analysis will develop knowledge and skills that may be useful to these careers:
Data Scientist
A Data Scientist is responsible for developing and deploying machine learning models to solve business problems. This course provides a strong foundation in the principles of data analysis and machine learning, including data preparation, descriptive analytics, model training, and result interpretation. Additionally, the course covers a variety of data analysis techniques, such as Exploratory Data Analysis, Feature Screening, Segmentation, Association Rules, Nearest Neighbors, Clustering, Decision Tree, Linear Regression, Logistic Regression, and Performance Evaluation. These skills are essential for success in the field of data science.
Machine Learning Engineer
A Machine Learning Engineer is responsible for developing and deploying machine learning models to solve business problems. This course provides a strong foundation in the principles of data analysis and machine learning, including data preparation, descriptive analytics, model training, and result interpretation. Additionally, the course covers a variety of data analysis techniques, such as Exploratory Data Analysis, Feature Screening, Segmentation, Association Rules, Nearest Neighbors, Clustering, Decision Tree, Linear Regression, Logistic Regression, and Performance Evaluation. These skills are essential for success in the field of machine learning engineering.
Data Analyst
A Data Analyst is responsible for analyzing data to uncover insights that can help businesses make better decisions. This course provides a strong foundation in the principles of data analysis, including data preparation, descriptive analytics, model training, and result interpretation. Additionally, the course covers a variety of data analysis techniques, such as Exploratory Data Analysis, Feature Screening, Segmentation, Association Rules, Nearest Neighbors, Clustering, Decision Tree, Linear Regression, Logistic Regression, and Performance Evaluation. These skills are essential for success in the field of data analysis.
Statistician
A Statistician is responsible for collecting, analyzing, and interpreting data. This course provides a strong foundation in the principles of statistics, including data analysis, probability, and hypothesis testing. Additionally, the course covers a variety of statistical techniques, such as Exploratory Data Analysis, Feature Screening, Segmentation, Association Rules, Nearest Neighbors, Clustering, Decision Tree, Linear Regression, Logistic Regression, and Performance Evaluation. These skills are essential for success in the field of statistics.
Quantitative Analyst
A Quantitative Analyst is responsible for developing and deploying quantitative models to solve business problems. This course provides a strong foundation in the principles of data analysis and quantitative finance. Additionally, the course covers a variety of data analysis techniques, such as Exploratory Data Analysis, Feature Screening, Segmentation, Association Rules, Nearest Neighbors, Clustering, Decision Tree, Linear Regression, Logistic Regression, and Performance Evaluation. These skills are essential for success in the field of quantitative finance.
Operations Research Analyst
An Operations Research Analyst is responsible for developing and deploying mathematical models to solve business problems. This course provides a strong foundation in the principles of data analysis and operations research. Additionally, the course covers a variety of data analysis techniques, such as Exploratory Data Analysis, Feature Screening, Segmentation, Association Rules, Nearest Neighbors, Clustering, Decision Tree, Linear Regression, Logistic Regression, and Performance Evaluation. These skills are essential for success in the field of operations research.
Business Analyst
A Business Analyst is responsible for analyzing business problems and recommending solutions. This course provides a strong foundation in the principles of data analysis and business intelligence. Additionally, the course covers a variety of data analysis techniques, such as Exploratory Data Analysis, Feature Screening, Segmentation, Association Rules, Nearest Neighbors, Clustering, Decision Tree, Linear Regression, Logistic Regression, and Performance Evaluation. These skills are essential for success in the field of business analysis.
Risk Analyst
A Risk Analyst is responsible for identifying, assessing, and mitigating risks to businesses. This course provides a strong foundation in the principles of data analysis and risk management. Additionally, the course covers a variety of data analysis techniques, such as Exploratory Data Analysis, Feature Screening, Segmentation, Association Rules, Nearest Neighbors, Clustering, Decision Tree, Linear Regression, Logistic Regression, and Performance Evaluation. These skills are essential for success in the field of risk management.
Database Administrator
A Database Administrator is responsible for managing and maintaining databases. This course provides a strong foundation in the principles of database administration, including database design, database optimization, and database security. Additionally, the course covers a variety of database administration techniques, such as data warehousing, data mining, and data visualization. These skills are essential for success in the field of database administration.
Data Architect
A Data Architect is responsible for designing and implementing data management solutions. This course provides a strong foundation in the principles of data architecture, including data modeling, data governance, and data security. Additionally, the course covers a variety of data architecture techniques, such as data warehousing, data mining, and data visualization. These skills are essential for success in the field of data architecture.
Data Engineer
A Data Engineer is responsible for designing, building, and maintaining data pipelines. This course provides a strong foundation in the principles of data engineering, including data extraction, transformation, and loading. Additionally, the course covers a variety of data engineering techniques, such as data warehousing, data mining, and data visualization. These skills are essential for success in the field of data engineering.
Mobile App Developer
A Mobile App Developer is responsible for designing, developing, and testing mobile apps. This course provides a strong foundation in the principles of mobile app development, including mobile app design, mobile app development, and mobile app testing. Additionally, the course covers a variety of mobile app development techniques, such as data warehousing, data mining, and data visualization. These skills are essential for success in the field of mobile app development.
Software Engineer
A Software Engineer is responsible for designing, developing, and testing software. This course provides a strong foundation in the principles of software engineering, including software design, software development, and software testing. Additionally, the course covers a variety of software engineering techniques, such as data warehousing, data mining, and data visualization. These skills are essential for success in the field of software engineering.
Web Developer
A Web Developer is responsible for designing, developing, and maintaining websites. This course provides a strong foundation in the principles of web development, including web design, web development, and web testing. Additionally, the course covers a variety of web development techniques, such as data warehousing, data mining, and data visualization. These skills are essential for success in the field of web development.
Data Visualization Specialist
A Data Visualization Specialist is responsible for designing and developing data visualizations. This course provides a strong foundation in the principles of data visualization, including data visualization design, data visualization development, and data visualization testing. Additionally, the course covers a variety of data visualization techniques, such as data warehousing, data mining, and data visualization. These skills are essential for success in the field of data visualization.

Reading list

We've selected 13 books that we think will supplement your learning. Use these to develop background knowledge, enrich your coursework, and gain a deeper understanding of the topics covered in Data Preparation and Analysis.
A classic textbook that provides a comprehensive introduction to statistical learning methods, including supervised and unsupervised learning, as well as model selection and evaluation.
A comprehensive guide to using R for data science, covering topics such as data exploration, visualization, modeling, and communication.
A comprehensive textbook that covers a wide range of machine learning topics, including supervised and unsupervised learning, Bayesian methods, and kernel methods.
A textbook that provides a more theoretical and probabilistic perspective on machine learning, suitable for advanced students and researchers.
A textbook that provides a practical introduction to data science for business applications, covering topics such as data mining, predictive modeling, and decision making.
A textbook that focuses on the theoretical foundations of machine learning, covering topics such as optimization, generalization, and reinforcement learning.
A practical guide to exploratory data analysis using R, covering topics such as data visualization, data transformation, and statistical modeling.
A practical guide to data manipulation using R, covering topics such as data wrangling, data cleaning, and data transformation.
A practical guide to using Python for data analysis, covering topics such as data manipulation, visualization, and statistical modeling.
A practical guide to programming in R, covering topics such as data structures, functions, and object-oriented programming.

Share

Help others find this course page by sharing it with your friends and followers:

Similar courses

Here are nine courses similar to Data Preparation and Analysis.
Our mission

OpenCourser helps millions of learners each year. People visit us to learn workspace skills, ace their exams, and nurture their curiosity.

Our extensive catalog contains over 50,000 courses and twice as many books. Browse by search, by topic, or even by career interests. We'll match you to the right resources quickly.

Find this site helpful? Tell a friend about us.

Affiliate disclosure

We're supported by our community of learners. When you purchase or subscribe to courses and programs or purchase books, we may earn a commission from our partners.

Your purchases help us maintain our catalog and keep our servers humming without ads.

Thank you for supporting OpenCourser.

© 2016 - 2024 OpenCourser