Building Deep Learning Models on Databricks

In this course, you will learn to train neural network models using TensorFlow and PyTorch, perform distributed training using the Horovod framework, and perform hyperparameter tuning using Hyperopt.

The Databricks Data Lakehouse platform offers a managed environment to train and compare your deep learning models, perform hyperparameter tuning, and productionize and serve your models.

In this course, Building Deep Learning Models on Databricks, you will learn to use Bamboolib for no-code data analysis and transformations.

First, you will build deep learning models using TensorFlow 2.0 and Keras, and will create a workspace experiment to manage your runs and use autologging to track model parameters, metrics, and artifacts.

Next, you will compare multiple runs to find the best-performing model using the MLflow UI.

Then, you will see that in order to have support for autologging in MLflow you need to use the PyTorch Lightning framework to design and train your model. You will also register your model with the model registry and use it for batch inference, deploy a Classic MLflow endpoint to serve model predictions, and use the Horovod framework for distributed training of your model.

Finally, you will learn how you can use the Hyperopt tool for hyperparameter tuning of your deep learning models, and will run hyperparameter tuning in a distributed fashion on a Spark cluster using the SparkTrials class.

When you are finished with this course, you will have the skills and knowledge to build and train deep machine learning models on Databricks using MLflow to manage your machine learning workflow.