Save for later

Improving Deep Neural Networks

Hyperparameter tuning, Regularization and Optimization

This course is a part of Deep Learning, a 5-course Specialization series from Coursera.

This course will teach you the "magic" of getting deep learning to work well. Rather than the deep learning process being a black box, you will understand what drives performance, and be able to more systematically get good results. You will also learn TensorFlow. After 3 weeks, you will: - Understand industry best-practices for building deep learning applications. - Be able to effectively use the common neural network "tricks", including initialization, L2 and dropout regularization, Batch normalization, gradient checking, - Be able to implement and apply a variety of optimization algorithms, such as mini-batch gradient descent, Momentum, RMSprop and Adam, and check for their convergence. - Understand new best-practices for the deep learning era of how to set up train/dev/test sets and analyze bias/variance - Be able to implement a neural network in TensorFlow. This is the second course of the Deep Learning Specialization.

Get Details and Enroll Now

OpenCourser is an affiliate partner of Coursera.

Get a Reminder

Not ready to enroll yet? We'll send you an email reminder for this course

Send to:

Coursera

&

deeplearning.ai

Rating 4.8 based on 4,549 ratings
Length 4 weeks
Effort 3 weeks, 3-6 hours per week
Starts May 18 (last week)
Cost $49
From deeplearning.ai via Coursera
Instructors Andrew Ng, Head Teaching Assistant - Kian Katanforoosh, Teaching Assistant - Younes Bensouda Mourri
Download Videos On all desktop and mobile devices
Language English
Subjects Programming Data Science
Tags Computer Science Data Science Algorithms Machine Learning

Get a Reminder

Get an email reminder about this course

Send to:

Similar Courses

What people are saying

According to other learners, here's what you need to know

andrew ng in 279 reviews

Andrew Ng's simple explanations made learning and grasping new concepts easier than ever.

Andrew Ng is as always very precise about the issues presented and helps build up our knowledge step-by-step in a super structured way.

ike usual andrew ng perfect explanation simple go to essential stuff.the minus points some troubles with notebookbig thanks for andrew ng's team.

Course if full of rare intuitions you could get only from someone like Andrew Ng.

宝贵的超参数调试经验,谢谢分享 Prof. Andrew Ng is fantastic at explaining complex ideas in a systematic, easy to understand way.

Great course, Andrew Ng is just the best teacher for NN and machine learning.

Andrew Ng does a GREAT job of breaking things down and making it easy.

just astonishingly brilliant course by andrew ng and crew Course was concise and assignments were well guided.

Really well taught by Professor Andrew Ng.

Thank you Such a wonderful course, no one can the material simpler than Andrew Ng.

Andrew Ng and the teaching assistants' team of this class are obviously very very determined not to leave any single major subject in deep learning without coverage.

Thank you Andrew Ng.

Andrew Ng is really an excellent teacher, perfect pace and great course overall!

As always Prof Andrew Ng is brilliant in elucidating complex concepts and techniques in simple and intuitive language.

Read more

deep neural networks in 63 reviews

For the first time, I now have a better intuition for optimizing and tuning hyperparameters used for deep neural networks.I got motivated to learn more after completing this course.

More pragmatic approach with theorems would be more appealing....or maybe it is me as i'd prefer Java (DL4J)...not sure Overall very good course where you quickly gain an overview of current optimization techniques for Deep Neural Networks Fantastic course.

The course make sense of all you learned previously on deep neural networks.

It is really great and it help you to get starting to Tensorflow True to the claimed learning objectives, the course Improving Deep Neural Networks shows some of the magic behind deep learning algorithms.

Awesome course by awesome teacher :)) Great Course and great learning 好 Very informative course on tuning the deep neural networks.

Having introduced the building blocks of deep neural networks, in this course Andrew teaches more advanced and practical concepts - like: regularization, advanced optimization techniques, batch-normalization, etc - that can significantly improve the implementation of the models we build.Also, in this course we get to learn TensorFlow, a widely used and wonderful deep learning framework.I highly recommend this course.Thank you Andrew & Co. :-) Good demonstration It was very helpful to grasp the differences in the concept of 'Normalization, Regularization and Optimization' Thank you for clarifying the most relevant techniques for improving the methods.

Excellent course and explanations on fundamental concepts for improving deep neural networks.

Very useful guidelines to achieve successful results developing Deep Neural Networks Excellent course, even using intuitions Prof. Andrew Ng is able to communicate the very details of the different regularization approaches, as well how to do a good hyper-parameter search.

I this course I learned how to improve Deep Neural Networks by applying different methods that help to speed up the convergence and to reduce overfitting.

Awesome Course Very good course about more critical concepts when building deep neural networks.

Very very useful tips and tricks about the most frequently used techniques for improving the performance of deep neural networks.

Highly recommended if you know deep neural networks and willing to dive deeper into them.

A solid course in working with deep neural networks.

good but would have been great if tensorflow is covered more It is amazingly rewarding to learn from Andrew, who is able to articulate so much insights into so many complicated refinements of Deep Neural Networks from so many different research papers.

Read more

batch norm in 65 reviews

A lot of useful info regarding ADAM , RMSprop and Batch Norm.

Struggled a bit to grasp the batch nomalization, Initially Regularization was also hard to grasp the first time, subsequent viewing made it clear though but batch norm still is a bit hazy.

Covers batch norm and optimizers nicely.

Like I could not imagine how backward propagation should be implemented in batch norm.

Batch Norm blew my mind.

Good overview of batch normalization.

batch normalization, one-hot encoding, softmax classification) with synthetic data on the third week.

One star was taken off, as I would like to see more in-depth info on Batch Norm and a bit more discussion on how to compute gradients in case that is used.

I never had this deep understanding of tuning hyperparameters, batch normalization and regularization before taking this course, though I went through several online material.

Just one minor point imo that the tutorial on TensorFlow may need to go deeper for those techniques mentioned previously in this course, for instance implementing batch norm.

Gained insight into many things like Batch Normalization and L2 regularization!

well explained concepts on hyper-parameter tuning batch normalization and programming frameworks You could have focussed more on Tensor Flow.Concepts are explained very well.I loved this curse.

Also, my understanding of batch norm improved.

Really helped me gain a detailed understanding of optimization techniques such as RMSprop and Adam, as well as the inner workings of batch normalization.

Read more

tensor flow in 60 reviews

Good: Contents on Tensor FlowBad: No real useful content compared the Course 1.

Great introduction to hyperparameters adjustment and Tensor flow Excelent course, very useful.

The class should include more introduction on the current ml frameworks such as tensor flow etc.

But anyway, this course is already good enough A very nice course providing intuitions and concepts for tuning the hyper parameters in a neural network.Also, provides a taste of using Tensor Flow (Neural Network Framework) in a comprehensive manner.

this course is very good it gives good practical ways to implement neural networks in python and with Tensor Flow, i like it very much I REALLY ENJOYED IT !!!!

nice I would like to have a brief introduction to Tensor Flow or a simple beginners tutorial (at least to have it clearer the usage of variables, constants and placeholders) Practical with lots of helpful tips.

Nice introduction to hyper-parameter tuning and tensor flow.

Exceptional Course, the Hyper parameters explanations are excellent every tip and advice provided help me so much to build better models, I also really liked the introduction of Tensor Flow Thanks.

Tensor flow tutorial is excellent.

The instructions, tips made Tensor flow coding section to be easy .

Cleared up a lot of my confusion about tensor flow.

I, however have one suggestion, the Introduction to Tensor Flow looked quite fast and could have been done in a better way by giving more slides about TensorFlow and then going on to the examples.

Useful material and good teacher but the grading system has some serious issues This course was excellent, however the Tensor flow at the end feels a little bit like the ML field is quickly being overtaken by the frameworks, and the Tensor flow section is a little bit tacked onto this course, maybe in a hurry.

Tensor flow is a very important tool and its inclusion in this course is a plus.

Read more

batch normalization in 40 reviews

Grate course, only criticism is that week 3 didn't thoroughly explain how batch normalization parameters (gamma and beta) get updated during gradient descent.

It was very helpful to learnt batch normalization, regularization and tensorflow.

Batch Normalization4).

Some more advanced topics are presented that students don't typically learn on coursera courses, such as improvements to gradient descent, batch normalization, and dropout.

Contains very good understanding of Hyperparameters and their tuning process.Secondly, teaches very well the mathematics of optimizers such as GD, SGD, GD with Momentum, GD with RMSProp and ADAM.Finally, a small glimpse of Batch Normalization.Highly Recommended!!!!!!

Great Very good course, although it'd be awesome if Andrew went over the backprop associated with Batch Normalization and perhaps a programming example of using Batch norm on my test set.

A very well curse, For me the information of batch normalization is very important Content great!

This course is a depository of tools that helps your NN be more perfect.Batch normalization is pretty sophisticated topic.

However, as I am currently not utilizing large datasets, the material on Batch Normalization and Hyper-parameter tuning is not useful to me ( now and in the foreseeable future).

we didn't try to implement the batch normalization ourselves and to incorporate batch normalization with other parameters etc.

It would be nice to include more details about the implementation of batch normalization.

In this course I learned or reviewed the following items in enough depth to perform in a job with them:InitializationRegularizationBatch NormalizationOptimizersTensorFlow Really well structured course.

week 2 was extremely boring I particularly benefited from the explanations of dropout, batch normalization and the RMSProp/Adam optimisers.

Super good course, would be greater if the programming assignments can cover batch normalization part instead of just TensorFlow.

Read more

looking forward in 28 reviews

I am looking forward for more courses in this deep learning series.

Great Course Worth the time if you're interested in deep learning This is such a crucial course to build upon the fundamentals of Neural Networks.Especially the intuitions that Andrew has provided really add to the arsenal, I'm so glad I took this course.Looking forward to the other courses in this specialisation.Thank you Andrew/Coursera :) I was excited to learn TensorFlow and this course provides the foundation for that as well as continue the concepts from Course 1Thank you Clearly explains optimization algorithms and techniques to improve neural network performance.

Looking forward to some more great learning!

Looking forward to do more courses from the team.

Looking forward to the next course This course gives me a deep understanding of how to tune hyper parameter, how to implement regularization and optimization.

I'm looking forward to start with course 3.

非常棒的课程。 Looking forward to Convolutional Neural Networks and Sequence Models.

Looking forward to advance my knowledge and experience with the next courses!

I am looking forward for the complete series.

Looking forward to the next course!

Looking forward to the next courses.

Looking forward to complete specialization.

I'm very much looking forward to the remaining courses in the Specialization.

Looking forward to immediately learn the next course as I implement the knowledge.

Read more

deep learning specialization in 23 reviews

This was another fantastic course in the deep learning specialization.

While the first course in Deep Learning Specialization gives us the fundamental insight, this course shows us the practical aspects such as hyper-parameters tuning and mini-batching .

I look forward to learning more in the remaining courses of the Deep Learning Specialization!

Again a very good course in the deep learning specialization.

I am looking forward to finish Deep Learning specialization.

This course contains a bit more interesting topics and I did not feel it was as overlapping to the Ng Machine Lerning course as the first course in the Deep Learning specialization was.

A continuation of the Deep Learning specialization, this course actually teaches many of the latest ideas in hyperparameter optimization and builds up the ideas nicely.

Simply awesome In the second course of the Deep Learning specialization Andrew gets deeper into the different subjects of Neural Networks.

Great course, which is part of an amazing Deep Learning specialization.

The best deep learning specialization ever with simple and clear explanation.

Many many thanks for putting this great deep learning specialization together!!

very useful also for experts All the courses in the Deep Learning Specialization are very good and met my expectations.

This course is a big part of the meat of the Deep Learning specialization.

Muchas Gracias... !The course is very good, particularly I am very grateful to COURSERA, for giving me the opportunity to do the five courses of the Deep Learning Specialization with financial aid and allowing me to have access to this type of training and certification.

Read more

gradient descent in 25 reviews

And it's been the first time that I get to know so many variants of gradient descent method, such as Adam and RMSprop.

Very useful! Good explanation of gradient descent w/ momentum.

It also provides good explanation for different optimization algorithms (enhancements to stochastic gradient descent).

I've learnt much from course including preprocessing (mini-batch, regularization, normalization), gradient descent algorithm (batch gradient descent, stochastic gradient descent, mini-batch gradient descent) and the variants (momentum, RMSProp, Adam).

Some useful addition to the Stanford Course are briefs on Gradient Descent With Momentum, RMSdrop and Adam as well as elementary practices on Tensorflow.

Very good course in terms of material and explanation.The autograder for grading the problems is not very good:- it fails a lot due to "technical difficulties",- it sometimes fails even though the solution is correct, e.g., a = a + b succeeds but a += b fails,- It sometimes passes even though the solution is incorrect (I used gradient descent instead of Adam, but it still passed with a 100% score).There were also quite some typos and other errors in the last assignment, so it seems like it was made in a bit of a hurry.

4.5/5 A diferencia del primer curso que es una continuacion del de Machine Learning de Andrew Ng , aqui vemos una evolución del contenido , se pasa a ver miniBatch Gradient Descent, Regularizacion , Momentum , Adam , y un inicio a tensorflowrealmente un MUY BUEN Curso Maybe this course can merge with the 1st one.

Covers optimization algorithms, Minibatch Gradient Descent, with Momentum, Adam, Xavier initialization, etc.

Sigmoid, Tangenth, activations, and so forth, this will help you understand terms such as L2 regularization, gradient descent with momentum, RMSProp, Adam, Exponentially weighted averages, and many others.

Going beyond gradient descent, types of regularization, hyperparameter searching we get to a set of robust tools that quickly find good solutions in extremely high dimensional spaces.

Excellent course, updated my knowledge of gradient descent optimization techniques, introduced me to tensorflow.

the gradient descent, the chain rule ) are quite easy-understanding and clear to most people, how to choose the hyperparameter and how to effectively carry out the projects are real essence.

Highly recommended for an beginner deep learning practitioner I got a very good understanding about how to what are the parameters I can tune, regularization methods, and different gradient descent alternatives which can be used through this course.

Yet another amazing course Recommended course for understanding the importance of hyperparameters in Neural Networks and understanding the structure of the optimizers used for training (gradient descent to ADAM) NICE COURSE!It takes my a lot of time, but it indeed deserve our effort!

Read more

hyper parameter tuning in 19 reviews

Very Important course to learn Hyper parameter tuning Useful information, good intuition, but lack of formal results.

The examples shown are very intuitive, which I like very much Very good course with proper explanation of Hyper parameter tuning, Regularization and Optimization of deep learning Excellent detail, and the introduction to Tensor Flow it is really amazing in symplicity Again, nice videos but not Excellent course.

As this course has helped me learn about following topics.Bias/Variance tradeoff, Different types of regularization methods, Code optimization techniques to speed up learning weights, Different types of weight optimization algorithms , About Hyper parameter tuning, Method for normalizing activation as batch norm, About Multi class classification and An introduction to Tensorflow This is a great course and you get to do real programming and training of a Deep Neural network.

Detailed instruction on optimization, hyper parameter tuning, batch normalization and deep learning framework Very informative but got some issues with the last programming assignment.

Thank you for your teaching :) Excellent course, pretty useful learn about hyper parameter tuning... and the INTUITION ( :D ) can be applied into other ML algorithms :) .

This is very well designed course to understand the hyper parameter tuning in a systematic manner other to convege the model faster.

A very good course to know more about Optimization techniques and Hyper parameter tuning.

This course teaches you about the Hyper parameter tuning, Regularization and Optimization.

The focus on hyper parameter tuning is instructive.

hyper parameter tuning, and yet there were no exercises to grasp how we can tune (more than one) hyper parameters through programming exercise.

Very important course for understanding hyper parameter tuning and optimization A very good course.

There is very much piece of information on hyper parameter tuning that is difficult to self-teach.The introduction to TensorFlow in notebook is wery well done.I will soon continue with next course.

A project component can also be added to make it even better Provides a good understanding of hyper parameter tuning Again... it was a great course.

Great course on Hyper parameter tuning with all the low level details needed.

Read more

step by step in 10 reviews

Step by step about essential work of deep learning 有时课件中有些小错误,也能理解 Good! the lectures in this course seemed very packed and rushed, squeezing in a lot of content that felt skipped over instead of delving into the math a bit.

But all credit to the way course is laid out and the step by step method of progress along with strong conceptual explanation helps a lot.

With step by step teaching us a lot of useful skills to train our model much faster, Andrew starts to put more attention on practical field, and rather than giving us many equations, he as before likes to use some vivid examples for giving us an intuition, which I think is very helpful to understand those scientific words of computer science.

The tutorial of Tensorflow and coding assignment are step by step, smooth to follow.

Once again programming excercise is rather easy to pass as you are guided step by step so there is no space for serious mistakes.

The course structure is very well defined, with step by step to build technical foundations in the beginning and later using open source deep learning framework to connect all the pieces together.

This time , I finished Regularzation, I think this is a interesting experience, for you can implement your alg step by step, I get some magic(not black magic) alg, like RMS, momentum and Adam.

At last, the most fascinating is to construct Tensorflow, just like a pipeline, step by step , and every step was made by only one line, from forward (without backward) to the model, Tensorflow is really black magic.

The assignments are well designed to make the step by step understanding and exercise of the learning.

Very Good Course to understand Step by Step Hyperparameter tuning, Regularization and Optimization to improve Deep Neuaral Networks & Practical Assignments !

Read more

balance between theory in 7 reviews

A right balance between theory (you are required to code know the models and code them from scratch) and practice (you get an overview of the frameworks available out there to put your code into production quickly and efficiently; and time is spent on practical aspects of the training phase).

Excellent course with attention to detail and good balance between theory and practise.

The right balance between theory and practice with good hands-on examples you can exercise without boring details of language syntax...

Wery usefull and clear Excellent Course It will help me to find solutions in ANN implementations it was a very good content to start from scratch Very well balance between theory and hands-on assignments.

I thought it was a good balance between theory and practice.

Good balance between theory and practice, focusing on the impact of hyperparameters across the whole solution.

Read more

nuts and bolts in 6 reviews

Compared to my knowledge before the course, I now feel like I have a sound understanding of all the small nuts and bolts that work in a deep learning system.

It makes us learn the nuts and bolts of neural networks.

Very good course to understand the nuts and bolts behind the deep learning great journey into deep learning world Content was good.But the system that checks our submitted our code checks wrongly even when I wrote it correctly.In week 2 assignment, when I submitted the code, it gave many functions as wrong coded.

So, the blame is on me not coursera.Hopefully I would fit more in the Deep Learning world by finishing up the course ;) Very practical Excellent introduction to hyperparameter tuning and tensorflow A useful class delving into the nuts and bolts of building a reliable nn.

The topics were advanced and practical, I am impressed This is an extremely important course, as it deals with lots best practices and nuts and bolts of Deep Learning that is the result of years of expertise and hard to find elsewhere Exceptionally awesome course!

Read more

Careers

An overview of related careers and their average salaries in the US. Bars indicate income percentile.

Learning Services $59k

Computer Vision, Deep Learning Engineer $67k

Computer Vision & Deep Learning Engineer $67k

Deep Clean Sales Specialist $76k

Deep clean specialist $76k

Deep Learning Research Scientist $86k

Deep Learning Research Engineer $88k

Research Scientist - Deep Learning $91k

Senior Learning Specialist, Learning and Development $102k

Deep Learning R&D Engineer $127k

Learning Assitant $142k

Deep Submergence Systems Program Manager $157k

Reviews

Sorted by most helpful reviews first

Guest says:

For what this course costs, it's almost a no brainer. You can get the same information scattered across the internet or parts of this course in an expensive textbook, but don't. Andrew Ng has already gathered the very best methods and tools and placed them right in front of you to implement structurally sound NN projects. Strongly recommended.

Write a review

Your opinion matters. Tell us what you think.

Coursera

&

deeplearning.ai

Rating 4.8 based on 4,549 ratings
Length 4 weeks
Effort 3 weeks, 3-6 hours per week
Starts May 18 (last week)
Cost $49
From deeplearning.ai via Coursera
Instructors Andrew Ng, Head Teaching Assistant - Kian Katanforoosh, Teaching Assistant - Younes Bensouda Mourri
Download Videos On all desktop and mobile devices
Language English
Subjects Programming Data Science
Tags Computer Science Data Science Algorithms Machine Learning

Similar Courses

Sorted by relevance

Like this course?

Here's what to do next:

  • Save this course for later
  • Get more details from the course provider
  • Enroll in this course
Enroll Now