Ever wondered how AI technologies like OpenAI ChatGPT, GPT-4, DALL-E, Midjourney, and Stable Diffusion really work? In this course, you will learn the foundations of these groundbreaking applications.
This course is the next logical step in my deep learning, data science, and machine learning series. I’ve done a lot of courses about deep learning, and I just released a course about unsupervised learning, where I talked about clustering and density estimation. So what do you get when you put these 2 together? Unsupervised deep learning.
Read more
Ever wondered how AI technologies like OpenAI ChatGPT, GPT-4, DALL-E, Midjourney, and Stable Diffusion really work? In this course, you will learn the foundations of these groundbreaking applications.
This course is the next logical step in my deep learning, data science, and machine learning series. I’ve done a lot of courses about deep learning, and I just released a course about unsupervised learning, where I talked about clustering and density estimation. So what do you get when you put these 2 together? Unsupervised deep learning.
Ever wondered how AI technologies like OpenAI ChatGPT, GPT-4, DALL-E, Midjourney, and Stable Diffusion really work? In this course, you will learn the foundations of these groundbreaking applications.
This course is the next logical step in my deep learning, data science, and machine learning series. I’ve done a lot of courses about deep learning, and I just released a course about unsupervised learning, where I talked about clustering and density estimation. So what do you get when you put these 2 together? Unsupervised deep learning.
In these course we’ll start with some very basic stuff - principal components analysis (PCA), and a popular nonlinear dimensionality reduction technique known as t-SNE (t-distributed stochastic neighbor embedding).
Next, we’ll look at a special type of unsupervised neural network called the autoencoder. After describing how an autoencoder works, I’ll show you how you can link a bunch of them together to form a deep stack of autoencoders, that leads to better performance of a supervised deep neural network. Autoencoders are like a non-linear form of PCA.
Last, we’ll look at restricted Boltzmann machines (RBMs). These are yet another popular unsupervised neural network, that you can use in the same way as autoencoders to pretrain your supervised deep neural network. I’ll show you an interesting way of training restricted Boltzmann machines, known as Gibbs sampling, a special case of Markov Chain Monte Carlo, and I’ll demonstrate how even though this method is only a rough approximation, it still ends up reducing other cost functions, such as the one used for autoencoders. This method is also known as Contrastive Divergence or CD-k. As in physical systems, we define a concept called free energy and attempt to minimize this quantity.
Finally, we’ll bring all these concepts together and I’ll show you visually what happens when you use PCA and t-SNE on the features that the autoencoders and RBMs have learned, and we’ll see that even without labels the results suggest that a pattern has been found.
All the materials used in this course are FREE. Since this course is the 4th in the deep learning series, I will assume you already know calculus, linear algebra, and Python coding. You'll want to install Numpy, Theano, and Tensorflow for this course. These are essential items in your data analytics toolbox.
If you are interested in deep learning and you want to learn about modern deep learning developments beyond just plain backpropagation, including using unsupervised neural networks to interpret what features can be automatically and hierarchically learned in a deep learning system, this course is for you.
This course focuses on "how to build and understand", not just "how to use". Anyone can learn to use an API in 15 minutes after reading some documentation. It's not about "remembering facts", it's about "seeing for yourself" via experimentation. It will teach you how to visualize what's happening in the model internally. If you want more than just a superficial look at machine learning models, this course is for you.
"If you can't implement it, you don't understand it"
Or as the great physicist Richard Feynman said: "What I cannot create, I do not understand".
My courses are the ONLY courses where you will learn how to implement machine learning algorithms from scratch
Other courses will teach you how to plug in your data into a library, but do you really need help with 3 lines of code?
After doing the same thing with 10 datasets, you realize you didn't learn 10 things. You learned 1 thing, and just repeated the same 3 lines of code 10 times...
Suggested Prerequisites:
calculus
linear algebra
probability
Python coding: if/else, loops, lists, dicts, sets
Numpy coding: matrix and vector operations, loading a CSV file
can write a feedforward neural network in Theano or Tensorflow
WHAT ORDER SHOULD I TAKE YOUR COURSES IN?:
Check out the lecture "Machine Learning and AI Prerequisite Roadmap" (available in the FAQ of any of my courses, including the free Numpy course)
Here's a deal for you
UDEAFNULP2024
What's inside
Learning objectives
- Understand the theory behind principal components analysis (pca)
- Know why pca is useful for dimensionality reduction, visualization, de-correlation, and denoising
- Derive the pca algorithm by hand
- Write the code for pca
- Understand the theory behind t-sne
- Use t-sne in code
- Understand the limitations of pca and t-sne
- Understand the theory behind autoencoders
- Write an autoencoder in theano and tensorflow
- Understand how stacked autoencoders are used in deep learning
- Write a stacked denoising autoencoder in theano and tensorflow
- Understand the theory behind restricted boltzmann machines (rbms)
- Understand why rbms are hard to train
- Understand the contrastive divergence algorithm to train rbms
- Write your own rbm and deep belief network (dbn) in theano and tensorflow
- Visualize and interpret the features learned by autoencoders and rbms
- Understand important foundations for openai chatgpt, gpt-4, dall-e, midjourney, and stable diffusion
- Show more
- Show less
- Understand the theory behind principal components analysis (pca)
- Know why pca is useful for dimensionality reduction, visualization, de-correlation, and denoising
- Derive the pca algorithm by hand
- Write the code for pca
- Understand the theory behind t-sne
- Use t-sne in code
- Understand the limitations of pca and t-sne
- Understand the theory behind autoencoders
- Write an autoencoder in theano and tensorflow
- Understand how stacked autoencoders are used in deep learning
- Write a stacked denoising autoencoder in theano and tensorflow
- Understand the theory behind restricted boltzmann machines (rbms)
- Understand why rbms are hard to train
- Understand the contrastive divergence algorithm to train rbms
- Write your own rbm and deep belief network (dbn) in theano and tensorflow
- Visualize and interpret the features learned by autoencoders and rbms
- Understand important foundations for openai chatgpt, gpt-4, dall-e, midjourney, and stable diffusion
- Show more
- Show less
Syllabus
Understand how we will use unsupervised learning in a deep learning context
Introduction and Outline
How to Succeed in this Course
Where to get the code and data
Read more
Syllabus
We use SVD to visualize the words in book titles. You'll see how related words can be made to appear close together in 2 dimensions using the SVD transformation.
Good to know
Save this course
Save
Activities
Review Python
Show steps
Brush up on Python programming basics, such as data structures, loops, and functions, to ensure a smooth learning experience in the course.
Show steps
-
Go through your previous Python notes or online tutorials.
-
Solve a few coding exercises or practice problems.
Revisit calculus and linear algebra
Show steps
Reinforce your understanding of calculus and linear algebra, which are essential prerequisites for this course.
Browse courses on
Calculus
Show steps
-
Review your class notes or textbook for calculus and linear algebra.
-
Complete practice problems or online quizzes to test your understanding.
Read 'Deep Learning' by Ian Goodfellow et al.
Show steps
Gain a comprehensive understanding of the theoretical foundations and practical applications of deep learning, including unsupervised learning techniques.
View
Deep Learning
on Amazon
Show steps
-
Read through the relevant chapters on unsupervised learning.
-
Attend the course lectures to supplement your understanding.
-
Complete the book's practice exercises or online quizzes.
Five other activities
Expand to see all activities and additional details
Show all eight activities
Join study group
Show steps
Collaborate with fellow students to discuss course concepts, work on assignments together, and provide mutual support.
Show steps
-
Connect with other students through online forums or social media.
-
Set up regular meeting times for study sessions.
-
Take turns presenting and explaining concepts, and engage in group discussions.
Code from scratch
Show steps
Implement various models from scratch, such as PCA, t-SNE, and autoencoders, to develop a deeper understanding of their inner workings.
Show steps
-
Follow the course instructions and code up the provided algorithms.
-
Experiment with different parameters and observe the impact on the results.
-
Troubleshoot any errors you encounter and seek clarification if needed.
Visualize the results
Show steps
Create visual representations of your models' results, such as scatter plots, heatmaps, or dimensionality reduction plots, to enhance your understanding of the learned features and patterns.
Show steps
-
Choose appropriate visualization techniques for the specific model and data.
-
Use Python libraries like Matplotlib or Seaborn to generate the visualizations.
-
Interpret the visualizations and extract insights about the data.
Attend a workshop on deep learning
Show steps
Deepen your understanding of unsupervised deep learning by attending a workshop led by experts in the field.
Show steps
-
Research and identify relevant workshops in your area or online.
-
Register and attend the workshop, taking notes and asking questions.
-
Apply the knowledge gained from the workshop to your course work.
Participate in a Kaggle competition
Show steps
Challenge yourself by applying your unsupervised deep learning skills to real-world datasets and compete with other data scientists for recognition and prizes.
Show steps
-
Identify a Kaggle competition related to unsupervised learning.
-
Download the dataset and explore it.
-
Develop and implement a solution using unsupervised deep learning models.
-
Submit your results and track your progress on the leaderboard.
Review Python
Show steps
Brush up on Python programming basics, such as data structures, loops, and functions, to ensure a smooth learning experience in the course.
Show steps
- Go through your previous Python notes or online tutorials.
- Solve a few coding exercises or practice problems.
Revisit calculus and linear algebra
Show steps
Reinforce your understanding of calculus and linear algebra, which are essential prerequisites for this course.
Browse courses on
Calculus
Show steps
- Review your class notes or textbook for calculus and linear algebra.
- Complete practice problems or online quizzes to test your understanding.
Read 'Deep Learning' by Ian Goodfellow et al.
Show steps
Gain a comprehensive understanding of the theoretical foundations and practical applications of deep learning, including unsupervised learning techniques.
View
Deep Learning
on Amazon
Show steps
- Read through the relevant chapters on unsupervised learning.
- Attend the course lectures to supplement your understanding.
- Complete the book's practice exercises or online quizzes.
Join study group
Show steps
Collaborate with fellow students to discuss course concepts, work on assignments together, and provide mutual support.
Show steps
- Connect with other students through online forums or social media.
- Set up regular meeting times for study sessions.
- Take turns presenting and explaining concepts, and engage in group discussions.
Code from scratch
Show steps
Implement various models from scratch, such as PCA, t-SNE, and autoencoders, to develop a deeper understanding of their inner workings.
Show steps
- Follow the course instructions and code up the provided algorithms.
- Experiment with different parameters and observe the impact on the results.
- Troubleshoot any errors you encounter and seek clarification if needed.
Visualize the results
Show steps
Create visual representations of your models' results, such as scatter plots, heatmaps, or dimensionality reduction plots, to enhance your understanding of the learned features and patterns.
Show steps
- Choose appropriate visualization techniques for the specific model and data.
- Use Python libraries like Matplotlib or Seaborn to generate the visualizations.
- Interpret the visualizations and extract insights about the data.
Attend a workshop on deep learning
Show steps
Deepen your understanding of unsupervised deep learning by attending a workshop led by experts in the field.
Show steps
- Research and identify relevant workshops in your area or online.
- Register and attend the workshop, taking notes and asking questions.
- Apply the knowledge gained from the workshop to your course work.
Participate in a Kaggle competition
Show steps
Challenge yourself by applying your unsupervised deep learning skills to real-world datasets and compete with other data scientists for recognition and prizes.
Show steps
- Identify a Kaggle competition related to unsupervised learning.
- Download the dataset and explore it.
- Develop and implement a solution using unsupervised deep learning models.
- Submit your results and track your progress on the leaderboard.
Career center
AI Researcher
Machine Learning Engineer
Data Scientist
Statistician
Data Analyst
Software Engineer
Quantitative Analyst
Financial Analyst
Operations Research Analyst
Actuary
Data Engineer
Business Analyst
Consultant
Teacher
Product Manager
Reading list
Save
Save
Share
Similar courses
OpenCourser helps millions of learners each year. People visit us to learn workspace skills, ace their exams, and nurture their curiosity.
Our extensive catalog contains over 50,000 courses and twice as many books. Browse by search, by topic, or even by career interests. We'll match you to the right resources quickly.
Find this site helpful? Tell a friend about us.
We're supported by our community of learners. When you purchase or subscribe to courses and programs or purchase books, we may earn a commission from our partners.
Your purchases help us maintain our catalog and keep our servers humming without ads.
Thank you for supporting OpenCourser.
