Unsupervised Deep Learning in Python from Udemy

Ever wondered how AI technologies like OpenAI ChatGPT, GPT-4, DALL-E, Midjourney, and Stable Diffusion really work? In this course, you will learn the foundations of these groundbreaking applications.

This course is the next logical step in my deep learning, data science, and machine learning series. I’ve done a lot of courses about deep learning, and I just released a course about unsupervised learning, where I talked about clustering and density estimation. So what do you get when you put these 2 together? Unsupervised deep learning.

In these course we’ll start with some very basic stuff - principal components analysis (PCA), and a popular nonlinear dimensionality reduction technique known as t-SNE (t-distributed stochastic neighbor embedding).

Next, we’ll look at a special type of unsupervised neural network called the autoencoder. After describing how an autoencoder works, I’ll show you how you can link a bunch of them together to form a deep stack of autoencoders, that leads to better performance of a supervised deep neural network. Autoencoders are like a non-linear form of PCA.

Last, we’ll look at restricted Boltzmann machines (RBMs). These are yet another popular unsupervised neural network, that you can use in the same way as autoencoders to pretrain your supervised deep neural network. I’ll show you an interesting way of training restricted Boltzmann machines, known as Gibbs sampling, a special case of Markov Chain Monte Carlo, and I’ll demonstrate how even though this method is only a rough approximation, it still ends up reducing other cost functions, such as the one used for autoencoders. This method is also known as Contrastive Divergence or CD-k. As in physical systems, we define a concept called free energy and attempt to minimize this quantity.

Finally, we’ll bring all these concepts together and I’ll show you visually what happens when you use PCA and t-SNE on the features that the autoencoders and RBMs have learned, and we’ll see that even without labels the results suggest that a pattern has been found.

All the materials used in this course are FREE. Since this course is the 4th in the deep learning series, I will assume you already know calculus, linear algebra, and Python coding. You'll want to install Numpy, Theano, and Tensorflow for this course. These are essential items in your data analytics toolbox.

If you are interested in deep learning and you want to learn about modern deep learning developments beyond just plain backpropagation, including using unsupervised neural networks to interpret what features can be automatically and hierarchically learned in a deep learning system, this course is for you.

This course focuses on "how to build and understand", not just "how to use". Anyone can learn to use an API in 15 minutes after reading some documentation. It's not about "remembering facts", it's about "seeing for yourself" via experimentation. It will teach you how to visualize what's happening in the model internally. If you want more than just a superficial look at machine learning models, this course is for you.

"If you can't implement it, you don't understand it"

Or as the great physicist Richard Feynman said: "What I cannot create, I do not understand".
My courses are the ONLY courses where you will learn how to implement machine learning algorithms from scratch
Other courses will teach you how to plug in your data into a library, but do you really need help with 3 lines of code?
After doing the same thing with 10 datasets, you realize you didn't learn 10 things. You learned 1 thing, and just repeated the same 3 lines of code 10 times...

Suggested Prerequisites:

calculus
linear algebra
probability
Python coding: if/else, loops, lists, dicts, sets
Numpy coding: matrix and vector operations, loading a CSV file
can write a feedforward neural network in Theano or Tensorflow

WHAT ORDER SHOULD I TAKE YOUR COURSES IN?:

Check out the lecture "Machine Learning and AI Prerequisite Roadmap" (available in the FAQ of any of my courses, including the free Numpy course)

What's inside

Learning objectives

Understand the theory behind principal components analysis (pca)
Know why pca is useful for dimensionality reduction, visualization, de-correlation, and denoising
Derive the pca algorithm by hand
Write the code for pca
Understand the theory behind t-sne
Use t-sne in code
Understand the limitations of pca and t-sne
Understand the theory behind autoencoders
Write an autoencoder in theano and tensorflow
Understand how stacked autoencoders are used in deep learning

Write a stacked denoising autoencoder in theano and tensorflow
Understand the theory behind restricted boltzmann machines (rbms)
Understand why rbms are hard to train
Understand the contrastive divergence algorithm to train rbms
Write your own rbm and deep belief network (dbn) in theano and tensorflow
Visualize and interpret the features learned by autoencoders and rbms
Understand important foundations for openai chatgpt, gpt-4, dall-e, midjourney, and stable diffusion
Show more
Show less

Understand the theory behind principal components analysis (pca)
Know why pca is useful for dimensionality reduction, visualization, de-correlation, and denoising
Derive the pca algorithm by hand
Write the code for pca
Understand the theory behind t-sne
Use t-sne in code
Understand the limitations of pca and t-sne
Understand the theory behind autoencoders
Write an autoencoder in theano and tensorflow
Understand how stacked autoencoders are used in deep learning
Write a stacked denoising autoencoder in theano and tensorflow
Understand the theory behind restricted boltzmann machines (rbms)
Understand why rbms are hard to train
Understand the contrastive divergence algorithm to train rbms
Write your own rbm and deep belief network (dbn) in theano and tensorflow
Visualize and interpret the features learned by autoencoders and rbms
Understand important foundations for openai chatgpt, gpt-4, dall-e, midjourney, and stable diffusion
Show more
Show less

Syllabus

Understand how we will use unsupervised learning in a deep learning context

Introduction and Outline

How to Succeed in this Course

Where to get the code and data

Tensorflow or Theano - Your Choice!

What are the practical applications of unsupervised deep learning?

Where does this course fit into your deep learning studies?

Principal Components Analysis

What does PCA do?

How does PCA work?

Why does PCA work? (PCA derivation)

PCA only rotates

MNIST visualization, finding the optimal number of principal components

PCA implementation

PCA for NLP

PCA objective function

PCA Application: Naive Bayes

SVD (Singular Value Decomposition)

Suggestion Box

t-SNE (t-distributed Stochastic Neighbor Embedding)

t-SNE Theory

t-SNE Visualization

t-SNE on the Donut

t-SNE on XOR

t-SNE on MNIST

Write your own autoencoder, and learn how to use stacked autoencoders to improve neural network training.

Autoencoders

Denoising Autoencoders

Stacked Autoencoders

Writing the autoencoder class in code (Theano)

Testing our Autoencoder (Theano)

Writing the deep neural network class in code (Theano)

Autoencoder in Code (Tensorflow)

Testing greedy layer-wise autoencoder training vs. pure backpropagation

Cross Entropy vs. KL Divergence

Deep Autoencoder Visualization Description

Deep Autoencoder Visualization in Code

An Autoencoder in 1 Line of Code

Understand the theory behind the RBM, what function it optimizes, and how to use it in a deep neural network. You will also learn how to build a completely unsupervised deep belief network (DBN).

Basic Outline for RBMs

Introduction to RBMs

Motivation Behind RBMs

Intractability

Neural Network Equations

Training an RBM (part 1)

Training an RBM (part 2)

Training an RBM (part 3) - Free Energy

RBM Greedy Layer-Wise Pretraining

RBM in Code (Theano) with Greedy Layer-Wise Training on MNIST

RBM in Code (Tensorflow)

The Vanishing Gradient Problem

The Vanishing Gradient Problem Description

The Vanishing Gradient Problem Demo in Code

Extras + Visualizing what features a neural network has learned

Exercises on feature visualization and interpretation

Understand how PCA applies to Latent Semantic Analysis or Latent Semantic Indexing

We use SVD to visualize the words in book titles. You'll see how related words can be made to appear close together in 2 dimensions using the SVD transformation.

Latent Semantic Analysis in Code

Application of t-SNE + K-Means: Finding Clusters of Related Words

Applications to Recommender Systems

Recommender Systems Section Introduction

Why Autoencoders and RBMs work

Data Preparation and Logistics

Data Preprocessing Code

AutoRec

AutoRec in Code

Categorical RBM for Recommender System Ratings

Recommender RBM Code pt 1

Recommender RBM Code pt 2

Recommender RBM Code pt 3

Recommender RBM Code Speedup

Theano and Tensorflow Basics Review

(Review) Theano Basics

(Review) Theano Neural Network in Code

(Review) Tensorflow Basics

(Review) Tensorflow Neural Network in Code

Setting Up Your Environment (FAQ by Student Request)

Pre-Installation Check

Anaconda Environment Setup

How to install Numpy, Scipy, Matplotlib, Pandas, IPython, Theano, and TensorFlow

Extra Help With Python Coding for Beginners (FAQ by Student Request)

How to Code by Yourself (part 1)

How to Code by Yourself (part 2)

Proof that using Jupyter Notebook is the same as not using it

Python 2 vs Python 3

Is Theano Dead?

Effective Learning Strategies for Machine Learning (FAQ by Student Request)

How to Succeed in this Course (Long Version)

Is this for Beginners or Experts? Academic or Practical? Fast or slow-paced?

Machine Learning and AI Prerequisite Roadmap (pt 1)

Machine Learning and AI Prerequisite Roadmap (pt 2)

Misc. topics tangentially related to the course that may help you with the course materials

What is the Appendix?

BONUS

Good to know

Know what's good

, what to watch for

, and possible dealbreakers

Develops advanced machine learning skills including dimensionality reduction, feature extraction, and unsupervised learning in neural networks

Explores bleeding-edge machine learning algorithms, such as OpenAI's ChatGPT, GPT-4, DALL-E, and Stable Diffusion

Taught by Lazy Programmer Inc. and Lazy Programmer Team, recognized for their expertise in machine learning and deep learning

Provides a strong foundation for students interested in pursuing careers in data science, machine learning, or deep learning

Requires students to have a strong background in calculus, linear algebra, and Python coding, which may be a barrier for some learners

Activities

Be better prepared before your course. Deepen your understanding during and after it. Supplement your coursework and achieve mastery of the topics covered in Unsupervised Deep Learning in Python with these activities:

Review Python

Show steps

Brush up on Python programming basics, such as data structures, loops, and functions, to ensure a smooth learning experience in the course.

Show steps

Go through your previous Python notes or online tutorials.
Solve a few coding exercises or practice problems.

Revisit calculus and linear algebra

Show steps

Reinforce your understanding of calculus and linear algebra, which are essential prerequisites for this course.

Browse courses on Calculus

Show steps

Review your class notes or textbook for calculus and linear algebra.
Complete practice problems or online quizzes to test your understanding.

Read 'Deep Learning' by Ian Goodfellow et al.

Show steps

Gain a comprehensive understanding of the theoretical foundations and practical applications of deep learning, including unsupervised learning techniques.

View Deep Learning on Amazon

Show steps

Read through the relevant chapters on unsupervised learning.
Attend the course lectures to supplement your understanding.
Complete the book's practice exercises or online quizzes.

Five other activities

Expand to see all activities and additional details

Show all eight activities

Join study group

Show steps

Collaborate with fellow students to discuss course concepts, work on assignments together, and provide mutual support.

Show steps

Connect with other students through online forums or social media.
Set up regular meeting times for study sessions.
Take turns presenting and explaining concepts, and engage in group discussions.

Code from scratch

Show steps

Implement various models from scratch, such as PCA, t-SNE, and autoencoders, to develop a deeper understanding of their inner workings.

Show steps

Follow the course instructions and code up the provided algorithms.
Experiment with different parameters and observe the impact on the results.
Troubleshoot any errors you encounter and seek clarification if needed.

Visualize the results

Show steps

Create visual representations of your models' results, such as scatter plots, heatmaps, or dimensionality reduction plots, to enhance your understanding of the learned features and patterns.

Show steps

Choose appropriate visualization techniques for the specific model and data.
Use Python libraries like Matplotlib or Seaborn to generate the visualizations.
Interpret the visualizations and extract insights about the data.

Attend a workshop on deep learning

Show steps

Deepen your understanding of unsupervised deep learning by attending a workshop led by experts in the field.

Show steps

Research and identify relevant workshops in your area or online.
Register and attend the workshop, taking notes and asking questions.
Apply the knowledge gained from the workshop to your course work.

Participate in a Kaggle competition

Show steps

Challenge yourself by applying your unsupervised deep learning skills to real-world datasets and compete with other data scientists for recognition and prizes.

Show steps

Identify a Kaggle competition related to unsupervised learning.
Download the dataset and explore it.
Develop and implement a solution using unsupervised deep learning models.
Submit your results and track your progress on the leaderboard.

Career center

Learners who complete Unsupervised Deep Learning in Python will develop knowledge and skills that may be useful to these careers:

AI Researcher

AI Researchers are at the forefront of developing and improving new unsupervised deep learning algorithms and techniques. They may use unsupervised deep learning to analyze large datasets, discover patterns, and develop new AI applications. This course's deep dive into PCA, Autoencoders, and RBMs will give an AI Researcher a strong background in some of the most important unsupervised deep learning techniques.

See salaries and explore the career path for AI Researcher

Machine Learning Engineer

Machine Learning Engineers directly apply techniques from this course to build artificial intelligence solutions to a variety of real-world problems. To wit, they improve decision making by automating it, increasing accuracy, and increasing efficiency. This course can help build a deep foundation for a Machine Learning Engineer - in particular, its topics in dimensionality reduction, autoencoders, and restricted Boltzmann machines would be of much use in the field. This course is an excellent way to prepare for a career as a Machine Learning Engineer, or to take on more responsibility in that role.

See salaries and explore the career path for Machine Learning Engineer

Data Scientist

Data Scientists utilize methods from this course to uncover patterns and trends in large volumes of data. They use unsupervised deep learning techniques to extract insights that may not be discovered through traditional methods. This course can provide the necessary theoretical background and practical skills for a Data Scientist in unsupervised deep learning, making it a valuable addition to a Data Scientist's toolkit.

See salaries and explore the career path for Data Scientist

Statistician

Statisticians who work in fields such as data science, machine learning, and artificial intelligence may find unsupervised deep learning techniques to be useful for analyzing complex data and extracting meaningful insights. This course covers the theoretical foundations and practical applications of such techniques, making it a valuable resource for Statisticians who wish to expand their skillset and contribute to cutting-edge research and development.

See salaries and explore the career path for Statistician

Data Analyst

Data Analysts may use techniques introduced in this course in order to prepare data for deeper analysis. They also use dimensionality reduction to explore and analyze relationships within complex datasets. Understanding the concepts behind PCA, Autoencoders, and RBMs will aid a Data Analyst in structuring datasets in a way that fosters discovery of patterns and insights.

See salaries and explore the career path for Data Analyst

Software Engineer

Software Engineers who wish to specialize in deep learning may find the techniques learned in this course to be particularly useful. Software Engineers who work on AI and machine learning projects may leverage unsupervised deep learning techniques to extract patterns and insights from data, improve model performance, and create more accurate and efficient AI systems.

See salaries and explore the career path for Software Engineer

Quantitative Analyst

Quantitative Analysts may use techniques from this course to uncover patterns and trends in financial data. They use unsupervised deep learning techniques to identify trading opportunities and make investment decisions. This course in unsupervised deep learning can provide the necessary quantitative foundation and practical skills. The course is particularly relevant for Quantitative Analysts who use advanced statistical models to uncover insights from data or make predictions.

See salaries and explore the career path for Quantitative Analyst

Financial Analyst

Financial Analysts may use unsupervised deep learning techniques to analyze financial data and make investment decisions. This course provides a strong foundation in the theory and practice of unsupervised deep learning, which can be applied to a wide range of financial applications.

See salaries and explore the career path for Financial Analyst

Operations Research Analyst

Operations Research Analysts who know unsupervised deep learning may use these methods to solve complex problems in a variety of industries. They use unsupervised deep learning techniques to analyze data, identify patterns, and develop solutions. This course's focus on PCA, Autoencoders, and RBMs will be particularly relevant for Operations Research Analysts who wish to apply these techniques to their work.

See salaries and explore the career path for Operations Research Analyst

Actuary

Actuaries who specialize in data science or machine learning may find unsupervised deep learning techniques to be useful for analyzing complex data and developing predictive models. This course provides a solid foundation in the theory and practice of unsupervised deep learning, which can give Actuaries a competitive edge in the field.

See salaries and explore the career path for Actuary

Data Engineer

Data Engineers often work closely with Data Scientists and Machine Learning Engineers, and may use unsupervised deep learning techniques to prepare data for analysis and modeling. This course may help Data Engineers build a foundation in dimensionality reduction, autoencoders, and RBMs that will allow them to more effectively support data science and machine learning initiatives.

See salaries and explore the career path for Data Engineer

Business Analyst

Business Analysts may use unsupervised deep learning to help businesses make better decisions. They can use these techniques to identify patterns and trends in customer data, market data, and other sources of information. Business Analysts who take this course will gain a solid understanding of the theoretical foundations and practical applications of unsupervised deep learning.

See salaries and explore the career path for Business Analyst

Consultant

Consultants who specialize in data science or machine learning may find unsupervised deep learning techniques to be useful for solving complex problems for their clients. This course provides a strong foundation in the theory and practice of unsupervised deep learning, which will enable Consultants to provide innovative and effective solutions to a wide range of business challenges.

See salaries and explore the career path for Consultant

Teacher

Teachers who specialize in computer science, data science, or machine learning may find this course to be a valuable resource for enhancing their knowledge and skills in unsupervised deep learning. This course provides a comprehensive overview of the theory and practice of unsupervised deep learning, equipping Teachers with the expertise to effectively teach these concepts to their students.

See salaries and explore the career path for Teacher

Product Manager

Product Managers who have a background in unsupervised deep learning may be able to use these techniques to develop better products. They can use these techniques to understand customer needs, identify market trends, and make better decisions about product development. This course will help Product Managers understand how to use unsupervised deep learning techniques, which can allow them to gain a competitive edge.

See salaries and explore the career path for Product Manager