Getting Started with the Databricks Lakehouse Platform from Pluralsight

This course will teach you how the Data Lakehouse architecture brings you the best of both Data Lakes and Data Warehouses allowing you to meet your data needs for big data processing, SQL analytics, and machine learning in a single platform.

Organizations have long collected data in a variety of formats, structured, unstructured, and semi-structured data. However, working with data in different formats for different use-cases requires multiple platform data warehouses for structured data needed for business intelligence and data lakes for unstructured data needed for data science and machine learning. The Databricks data lakehouse architecture is an innovative paradigm that combines the flexibility and cost efficiency of a data lake with the reliability and features of a data warehouse.

In this course, Getting Started with the Databricks Lakehouse Platform, you will learn the importance of storing data in a centralized repository and how data lakes and data warehouses serve to solve different data-related problems. First, you’ll explore a variety of technologies in the analytics space and how the lakehouse platform encompasses their strengths while mitigating their limitations.

Next, you will understand the basic components that make up the architecture of a data lakehouse and how the Databricks Lakehouse Platform uses Delta Lakes to enable both SQL analytics and data science and machine learning using the same underlying data lake storage.

Finally, you will explore the Databricks Data Lakehouse on Microsoft Azure. You will build the data lakehouse, store data in Delta Tables, and access the same data using Apache Spark and SQL queries.

When you are finished with this course, you will be able to clearly articulate how the data lakehouse platform helps mitigate challenges with current data architectures and will know hands-on how you can set up and use the lakehouse on Databricks.

What's inside

Syllabus

Course Overview

Introducing the Lakehouse Platform

An Architectural Overview of the Lakehouse Platform

Using a Lakehouse on Databricks

Good to know

Know what's good

, what to watch for

, and possible dealbreakers

Explored through the lens of a real-world use case, this course clarifies the technical details of the Data Lakehouse concept

Regardless of your learning objectives, this course provides a comprehensive understanding of data lakehouse architecture and will help you understand how data lakes and data warehouses work

Develops competency in using Delta Lakes, SQL, Apache Spark, and related technologies, enabling you to use and apply them in practical situations beyond this course

Taught by Janani Ravi, this course benefits from the instructor's experience and expertise in data engineering and data architecture

Examines the challenges of current data architectures, explaining how the data lakehouse platform mitigates them

Provides an opportunity to build and use a data lakehouse on Databricks, offering hands-on experience with the platform

Activities

Be better prepared before your course. Deepen your understanding during and after it. Supplement your coursework and achieve mastery of the topics covered in Getting Started with the Databricks Lakehouse Platform with these activities:

Review relational database management systems

Show steps

Review key concepts like table structures, SQL queries, and relational algebra to understand the differences and similarities between data warehouses and data lakes.

Browse courses on Relational Database Management Systems

Show steps

Review notes and textbooks from previous database courses
Do practice exercises on SQL queries
Read articles and tutorials on relational database concepts

Review data lake concepts

Show steps

Reinforce your understanding of data lakes, their purpose, and how they differ from traditional data warehouses.

Browse courses on Data Lakes

Show steps

Read articles and blog posts on data lakes
Review online tutorials on data lake architecture
Attend a webinar or online course on data lakes

Review Delta Lake Fundamentals

Show steps

Reviewing the basics of Delta Lakes will help reinforce your foundational understanding of the technology, ensuring a smoother transition into more complex concepts.

Browse courses on Data Lakes

Show steps

Go through online tutorials on Delta Lakes fundamentals.
Read documentation and articles on Delta Lakes architecture and concepts.
Set up a sandbox environment and practice creating and managing Delta tables.

20 other activities

Expand to see all activities and additional details

Show all 23 activities

Review data management and analytics concepts

Show steps

Refreshes foundational knowledge and prepares you for the course.

Browse courses on Data Management

Show steps

Review textbooks
Take practice quizzes

Review SQL basics

Show steps

Revisit and solidify your understanding of SQL to prepare for this course's data analytics and processing focus.

Browse courses on SQL

Show steps

Consult online tutorials or documentation to refresh your knowledge of SQL syntax and commands.
Practice writing basic SQL queries to retrieve, filter, and aggregate data from a sample database or dataset.

Join a Study Group or Discussion Forum

Show steps

Engaging with peers in study groups or discussion forums fosters a supportive learning environment, allowing you to exchange ideas, clarify concepts, and gain diverse perspectives.

Browse courses on Collaboration

Show steps

Identify online or in-person study groups or discussion forums related to the course topic.
Participate actively in discussions, sharing your insights and asking questions.
Collaborate on problem-solving and knowledge-sharing activities.
Attend virtual or in-person meetups or events to connect with other learners.

Organize and summarize course materials

Show steps

Helps you prepare for the course by reviewing and organizing the materials.

Show steps

Review the syllabus
Organize your notes

Read 'The Data Warehouse Toolkit'

Show steps

Provides a strong foundation for understanding the fundamental concepts of data warehousing.

View Kimball's Data Warehouse Toolkit Classics, 3... on Amazon

Show steps

Read the first 5 chapters
Complete the exercises in each chapter

Explore Databricks Lakehouse Platform tutorials

Show steps

Supplement your understanding of the Databricks Lakehouse Platform by following guided tutorials to build hands-on experience with its features and capabilities.

Show steps

Access the Databricks documentation or online tutorials for the Lakehouse Platform.
Follow a tutorial that aligns with your learning goals, such as building a data lake or performing SQL analytics on a data lake.
Implement the tutorial steps to gain practical experience with the platform's features.

Explore Apache Spark and SQL for Databricks

Show steps

Getting hands-on experience with Apache Spark and SQL will provide practical insights and enhance your ability to work with data in the Databricks Lakehouse Platform.

Browse courses on Apache Spark

Show steps

Follow step-by-step guides on setting up Apache Spark and Databricks SQL.
Complete tutorials on data manipulation, transformations, and visualizations.
Experiment with sample datasets and build simple data pipelines.
Explore community forums and resources for additional support.

Practice using Delta Lakes

Show steps

Develop hands-on experience working with Delta Lakes, the foundation of the Databricks Lakehouse Platform.

Browse courses on Delta Lakes

Show steps

Follow online tutorials on Delta Lakes
Complete exercises in the Databricks Academy on Delta Lakes

Create a Data Lakehouse Platform

Show steps

Provides hands-on experience in setting up and using a data lakehouse platform.

Show steps

Choose a cloud provider and provision a cluster
Configure the data lakehouse platform
Load data into the data lake

Follow guided tutorials on Databricks

Show steps

Provides step-by-step instructions for using Databricks to build data lakehouse solutions.

Show steps

Identify relevant tutorials
Complete the tutorials

Practice data manipulation and analysis with Delta Lake

Show steps

Develop proficiency in using Delta Lake for data manipulation and analysis tasks, which are core to the Databricks Lakehouse Platform.

Browse courses on Delta Lake

Show steps

Set up a Delta Lake environment on your local machine or a cloud platform.
Load a sample dataset into a Delta Lake table.
Perform data manipulations such as filtering, sorting, and aggregations using SQL or Apache Spark.
Analyze the results and draw insights from the data.

Attend a Databricks Lakehouse Platform workshop

Show steps

Enhance your knowledge and skills by participating in a workshop conducted by Databricks experts, providing you with hands-on experience and insights from industry professionals.

Show steps

Identify and register for a relevant Databricks Lakehouse Platform workshop.
Attend the workshop and actively participate in the sessions.
Engage with the instructors and other attendees to exchange ideas and learn from their experiences.

Complete Databricks Labs and Exercises

Show steps

Working through Databricks labs and exercises provides a structured and practical way to solidify your understanding and develop your skills in using the platform.

Browse courses on Data Analytics

Show steps

Complete the Databricks Labs associated with the course content.
Attempt additional exercises and challenges to test your knowledge.
Review solutions and explanations to reinforce your understanding.
Seek assistance from the Databricks community or support team if needed.

Write a blog post about the Data Lakehouse Architecture

Show steps

Helps you understand the key concepts and benefits of the data lakehouse architecture.

Show steps

Research the topic
Outline the key points
Write the first draft

Solve SQL and Apache Spark coding challenges

Show steps

Sharpens your SQL and Apache Spark skills for data processing and analysis.

Show steps

Find coding challenges
Solve the coding challenges

Build a simple data lakehouse

Show steps

Apply your learning by building a basic data lakehouse on the Databricks platform.

Browse courses on Data Lakehouse Architecture

Show steps

Create a Databricks account
Create a data lakehouse workspace
Ingest data into your data lakehouse
Create a Delta table
Query your data using SQL and Apache Spark

Build a mini data lakehouse project

Show steps

Apply your knowledge by constructing a miniature data lakehouse project that incorporates the concepts learned in this course, reinforcing your understanding of the platform.

Show steps

Define the scope and objectives of your mini project.
Gather and prepare a small dataset relevant to your project.
Create a data lakehouse on the Databricks platform.
Ingest the dataset into the data lakehouse and perform necessary data transformations.
Build a simple data pipeline to process and analyze the data.
Visualize and interpret the results to gain insights.

Attend meetups and webinars on data lakehouse

Show steps

Provides opportunities to connect with professionals and learn about industry trends.

Show steps

Identify relevant events
Attend the events

Participate in data lakehouse workshops

Show steps

Provides hands-on training from experts in the data lakehouse domain.

Show steps

Identify relevant workshops
Attend the workshops

Build a Mini Data Lakehouse Project

Show steps

Creating a small-scale data lakehouse project will allow you to apply your learnings in a practical setting, enhancing your understanding of the platform's capabilities.

Browse courses on Data Engineering

Show steps

Define the scope and objectives of your project.
Gather and prepare a small dataset that aligns with your project goals.
Set up a development environment and create a data lakehouse using Databricks.
Ingest and process the data into Delta tables.
Develop data pipelines and visualizations to analyze and present the data.

Career center

Learners who complete Getting Started with the Databricks Lakehouse Platform will develop knowledge and skills that may be useful to these careers:

Data Engineer

Data Engineers design, build, and maintain data pipelines and infrastructure to support data-driven applications and analytics. This course provides a hands-on introduction to the Databricks Lakehouse Platform, which is a leading platform for building and managing data lakehouses. By completing this course, Data Engineers can gain the skills and knowledge needed to build and maintain data lakehouses that support the data needs of their organizations.

See salaries and explore the career path for Data Engineer

Data Architect

Data Architects design, build, and manage data architectures and systems to meet the needs of an organization. This course provides a solid foundation for understanding the concepts and technologies involved in data lakehouse architecture, which is a key component of modern data architectures. By understanding the principles of data lakehouse architecture, Data Architects can make informed decisions about how to design and implement data systems that are scalable, reliable, and cost-effective.

See salaries and explore the career path for Data Architect

Data Scientist

Data Scientists use data to build models and insights to support decision-making. This course provides a solid foundation for understanding the role of data lakehouses in supporting data science and machine learning. By understanding the principles of data lakehouse architecture, Data Scientists can make informed decisions about how to access and use data for their projects.

See salaries and explore the career path for Data Scientist

Data Analyst

Data Analysts analyze data to identify trends and insights to support decision-making. This course provides a strong foundation for understanding the role of data lakehouses in supporting data analytics. By understanding the principles of data lakehouse architecture, Data Analysts can make informed decisions about how to access and use data for their analyses.

See salaries and explore the career path for Data Analyst

Business Intelligence Analyst

Business Intelligence Analysts use data to identify trends and insights to support business decision-making. This course provides a solid foundation for understanding the role of data lakehouses in supporting business intelligence. By understanding the principles of data lakehouse architecture, Business Intelligence Analysts can make informed decisions about how to access and use data for their analyses.

See salaries and explore the career path for Business Intelligence Analyst

Database Administrator

Database Administrators design, build, and maintain databases to support the data needs of an organization. This course provides a solid foundation for understanding the concepts and technologies involved in data lakehouse architecture, which is a key component of modern data architectures. By understanding the principles of data lakehouse architecture, Database Administrators can make informed decisions about how to design and implement data systems that are scalable, reliable, and cost-effective.

See salaries and explore the career path for Database Administrator

Software Engineer

Software Engineers design, build, and maintain software applications to support the needs of an organization. This course provides a solid foundation for understanding the concepts and technologies involved in data lakehouse architecture, which is a key component of modern data architectures. By understanding the principles of data lakehouse architecture, Software Engineers can make informed decisions about how to design and implement software applications that are scalable, reliable, and cost-effective.

See salaries and explore the career path for Software Engineer

Cloud Architect

Cloud Architects design, build, and manage cloud computing environments to support the needs of an organization. This course provides a solid foundation for understanding the concepts and technologies involved in data lakehouse architecture, which is a key component of modern data architectures. By understanding the principles of data lakehouse architecture, Cloud Architects can make informed decisions about how to design and implement cloud computing environments that are scalable, reliable, and cost-effective.

See salaries and explore the career path for Cloud Architect

Data Warehouse Architect

Data Warehouse Architects design, build, and maintain data warehouses to support the data needs of an organization. This course provides a solid foundation for understanding the concepts and technologies involved in data lakehouse architecture, which is a key component of modern data architectures. By understanding the principles of data lakehouse architecture, Data Warehouse Architects can make informed decisions about how to design and implement data warehouses that are scalable, reliable, and cost-effective.

See salaries and explore the career path for Data Warehouse Architect

Data Governance Specialist

Data Governance Specialists develop and implement policies and procedures to ensure the quality, accuracy, and security of data. This course provides a solid foundation for understanding the concepts and technologies involved in data lakehouse architecture, which is a key component of modern data architectures. By understanding the principles of data lakehouse architecture, Data Governance Specialists can make informed decisions about how to develop and implement data governance policies and procedures that are effective and efficient.

See salaries and explore the career path for Data Governance Specialist

Information Security Analyst

Information Security Analysts design, implement, and maintain security measures to protect data and information from unauthorized access or use. This course provides a solid foundation for understanding the concepts and technologies involved in data lakehouse architecture, which is a key component of modern data architectures. By understanding the principles of data lakehouse architecture, Information Security Analysts can make informed decisions about how to design and implement security measures that are effective and efficient.

See salaries and explore the career path for Information Security Analyst

Data Privacy Analyst

Data Privacy Analysts develop and implement policies and procedures to ensure that data is collected, used, and shared in a compliant and ethical manner. This course provides a solid foundation for understanding the concepts and technologies involved in data lakehouse architecture, which is a key component of modern data architectures. By understanding the principles of data lakehouse architecture, Data Privacy Analysts can make informed decisions about how to develop and implement data privacy policies and procedures that are effective and efficient.

See salaries and explore the career path for Data Privacy Analyst

Statistician

Statisticians collect, analyze, and interpret data to provide insights and make predictions. This course provides a strong foundation for understanding the role of data lakehouses in supporting statistical analysis. By understanding the principles of data lakehouse architecture, Statisticians can make informed decisions about how to access and use data for their analyses.

See salaries and explore the career path for Statistician

Operations Research Analyst

Operations Research Analysts use mathematical and analytical techniques to solve complex problems and improve decision-making. This course provides a solid foundation for understanding the role of data lakehouses in supporting operations research. By understanding the principles of data lakehouse architecture, Operations Research Analysts can make informed decisions about how to access and use data for their analyses.

See salaries and explore the career path for Operations Research Analyst

Financial Analyst

Financial Analysts use data to analyze financial performance and make investment recommendations. This course provides a solid foundation for understanding the role of data lakehouses in supporting financial analysis. By understanding the principles of data lakehouse architecture, Financial Analysts can make informed decisions about how to access and use data for their analyses.

See salaries and explore the career path for Financial Analyst