Getting Started with Delta Lake on Databricks from Pluralsight

This course will teach you how you can create, ingest data into, and work with Delta Lakes, an open-source storage layer that brings reliability to data stored in data lakes. Delta Lakes offer ACID transactions, unified batch and stream processing.

The Databricks Data Lakehouse architecture is an innovative paradigm that combines the flexibility and low-cost storage offered by data lakes with the features and capabilities of a data warehouse. The lakehouse architecture achieves this by using metadata, indexing, and caching layer on top of data lake storage. This open-source storage layer is Delta Lake. This Delta Lake storage layer lies at the heart of Databricks’ lakehouse architecture.

In this course, Getting Started with Delta Lake on Databricks you will learn how exactly Delta Lakes supports transactions on cloud storage. First, you will learn the basic elements of Delta Lake namely Delta files, Delta tables, DeltaLog, and Delta optimizations.

Next, you will discover how you can get better performance from queries that you run on Delta tables using different optimizations. Here you will explore Delta caching, data skipping, and file layout optimizations such as partitioning, bin-packing, and z-order clustering.

Finally, you will explore how you can ingest data from external sources into Delta tables using batch and streaming ingestion. You will use the COPY INTO command for batch ingestion and the Databricks Auto Loader for stream ingestion.

When you are finished with this course, you will have the skills and ability to create, and ingest data into Delta Lakes and run optimal queries to extract insights.

What's inside

Syllabus

Course Overview

Exploring Delta Lake on Databricks

Optimizing Queries on Delta Tables

Ingesting Batch and Streaming Data into Delta Tables

Good to know

Know what's good

, what to watch for

, and possible dealbreakers

Teaches using Databricks, industry-standard in data engineering

Covers batch ingestion, a crucial technique in working with data lakes

Taught by Janani Ravi, an expert in Delta Lake with extensive experience from working at AWS, Google, and Microsoft

Introduces Delta Lake optimizations, an essential element for improving processing on data lakes

Suitable for beginners who have prior knowledge of data lakes

Provides a foundation in Delta Lake concepts, such as Delta files and Delta optimizations, useful for beginners

Career center

Learners who complete Getting Started with Delta Lake on Databricks will develop knowledge and skills that may be useful to these careers:

Data Scientist

Data Scientists use data to build models that can predict future outcomes. This course may be useful for Data Scientists who want to learn how to use Delta Lake to improve the performance of their models.

See salaries and explore the career path for Data Scientist

Data Analyst

Data Analysts use data to identify trends and patterns. This course may be useful for Data Analysts who want to learn how to use Delta Lake to analyze large datasets more efficiently.

See salaries and explore the career path for Data Analyst

Data Engineer

A Data Engineer is responsible for designing, building, and maintaining data pipelines. This course may be useful for Data Engineers who want to learn how to use Delta Lake to create reliable and scalable data pipelines.

See salaries and explore the career path for Data Engineer

Big Data Architect

Big Data Architects design and implement data solutions for large-scale data systems. This course may be useful for Big Data Architects who want to learn how to use Delta Lake to build reliable and scalable data solutions.

See salaries and explore the career path for Big Data Architect

Cloud Engineer

Cloud Engineers design and implement cloud-based solutions. This course may be useful for Cloud Engineers who want to learn how to use Delta Lake to build reliable and scalable data solutions in the cloud.

See salaries and explore the career path for Cloud Engineer

Business Intelligence Analyst

Business Intelligence Analysts use data to make better business decisions. This course may be useful for Business Intelligence Analysts who want to learn how to use Delta Lake to analyze large datasets more efficiently.

See salaries and explore the career path for Business Intelligence Analyst

Database Administrator

Database Administrators maintain and manage databases. This course may be useful for Database Administrators who want to learn how to use Delta Lake to improve the performance and reliability of their databases.

See salaries and explore the career path for Database Administrator

Data Warehouse Engineer

Data Warehouse Engineers design and implement data warehouses. This course may be useful for Data Warehouse Engineers who want to learn how to use Delta Lake to build reliable and scalable data warehouses.

See salaries and explore the career path for Data Warehouse Engineer

Machine Learning Engineer

Machine Learning Engineers design and implement machine learning models. This course may be useful for Machine Learning Engineers who want to learn how to use Delta Lake to improve the performance and reliability of their machine learning models.

See salaries and explore the career path for Machine Learning Engineer

Statistician

Statisticians use data to draw conclusions about the world around us. This course may be useful for Statisticians who want to learn how to use Delta Lake to analyze large datasets more efficiently.

See salaries and explore the career path for Statistician

Data Architect

Data Architects design and implement data architectures. This course may be useful for Data Architects who want to learn how to use Delta Lake to build reliable and scalable data architectures.

See salaries and explore the career path for Data Architect

Software Engineer

Software Engineers design, develop, and maintain software applications. This course may be useful for Software Engineers who want to learn how to use Delta Lake to build reliable and scalable data pipelines.

See salaries and explore the career path for Software Engineer