In this course, you'll learn about the raw ingredients and processes that are used to physically store data on disk and in memory. You’ll explore different data systems, including object stores, block storage, file systems, and databases, that are built on top of these raw ingredients. You’ll also get a chance to query a Neo4j graph database and perform vector similarity search, a key feature behind generative AI and large language models. You will explore the evolution of data storage abstractions, from data warehouses to data lakes and data lakehouses, while comparing the advantages and drawbacks of each architectural paradigm.
Read more
In this course, you'll learn about the raw ingredients and processes that are used to physically store data on disk and in memory. You’ll explore different data systems, including object stores, block storage, file systems, and databases, that are built on top of these raw ingredients. You’ll also get a chance to query a Neo4j graph database and perform vector similarity search, a key feature behind generative AI and large language models. You will explore the evolution of data storage abstractions, from data warehouses to data lakes and data lakehouses, while comparing the advantages and drawbacks of each architectural paradigm.
In this course, you'll learn about the raw ingredients and processes that are used to physically store data on disk and in memory. You’ll explore different data systems, including object stores, block storage, file systems, and databases, that are built on top of these raw ingredients. You’ll also get a chance to query a Neo4j graph database and perform vector similarity search, a key feature behind generative AI and large language models. You will explore the evolution of data storage abstractions, from data warehouses to data lakes and data lakehouses, while comparing the advantages and drawbacks of each architectural paradigm.
With hands-on practice, you'll design a simple data lake using Amazon Glue, and build a data lakehouse using AWS LakeFormation and Apache Iceberg. In the last week of this course, you’ll see how queries work behind the scenes, practice writing more advanced SQL queries, compare the query performance in row vs column-oriented storage, and perform streaming queries using Apache Flink.
Good to know
Save this course
Save
Activities
Form a Study Group
Show steps
Engage with peers to discuss course concepts, review material, and support each other's learning.
Show steps
-
Work together on projects and assignments.
-
Find a group of students with complementary skills and interests.
-
Meet regularly to discuss the course material.
Data Wrangling Drills
Show steps
Reinforce your understanding of data wrangling techniques and strengthen your ability to work with data effectively.
Browse courses on
Data Wrangling
Show steps
-
Find a dataset and load it into a pandas DataFrame.
-
Perform data cleaning operations such as handling missing values, removing duplicates, and correcting data types.
-
Explore the data and identify patterns and insights.
Attend a Data Analytics Workshop
Show steps
Deepen your understanding of data analysis techniques and tools through hands-on workshops.
Browse courses on
Data Analytics
Show steps
-
Find a workshop that covers relevant topics.
-
Attend the workshop and participate actively.
-
Apply what you learn to your own projects.
Four other activities
Expand to see all activities and additional details
Show all seven activities
Intro to Apache Spark
Show steps
Expand your big data processing skills by exploring Apache Spark's capabilities for distributed computing.
Browse courses on
Apache Spark
Show steps
-
Follow tutorials on setting up and using Apache Spark.
-
Create a Spark DataFrame and perform basic transformations.
-
Explore Spark's RDDs and their usage for parallel processing.
Review 'Designing Data-Intensive Applications'
Show steps
Gain a comprehensive understanding of data-intensive application design and best practices.
View
Secret Colors
on Amazon
Show steps
-
Read chapters on data modeling, storage, and processing.
-
Identify key concepts and architectures for data-intensive applications.
Create a Data Glossary
Show steps
Strengthen your data management skills by creating a data glossary for a specific domain or project.
Show steps
-
Identify the relevant data terms and definitions.
-
Organize the terms into a structured and logical framework.
-
Publish the data glossary for reference and use.
Design a Data Lake
Show steps
Apply your knowledge of data storage and management by designing a data lake for a specific use case.
Browse courses on
Data Lake
Show steps
-
Define the requirements and scope of your data lake.
-
Choose appropriate data storage technologies and tools.
-
Design the data lake architecture and data pipeline.
Form a Study Group
Show steps
Engage with peers to discuss course concepts, review material, and support each other's learning.
Show steps
- Work together on projects and assignments.
- Find a group of students with complementary skills and interests.
- Meet regularly to discuss the course material.
Data Wrangling Drills
Show steps
Reinforce your understanding of data wrangling techniques and strengthen your ability to work with data effectively.
Browse courses on
Data Wrangling
Show steps
- Find a dataset and load it into a pandas DataFrame.
- Perform data cleaning operations such as handling missing values, removing duplicates, and correcting data types.
- Explore the data and identify patterns and insights.
Attend a Data Analytics Workshop
Show steps
Deepen your understanding of data analysis techniques and tools through hands-on workshops.
Browse courses on
Data Analytics
Show steps
- Find a workshop that covers relevant topics.
- Attend the workshop and participate actively.
- Apply what you learn to your own projects.
Intro to Apache Spark
Show steps
Expand your big data processing skills by exploring Apache Spark's capabilities for distributed computing.
Browse courses on
Apache Spark
Show steps
- Follow tutorials on setting up and using Apache Spark.
- Create a Spark DataFrame and perform basic transformations.
- Explore Spark's RDDs and their usage for parallel processing.
Review 'Designing Data-Intensive Applications'
Show steps
Gain a comprehensive understanding of data-intensive application design and best practices.
View
Secret Colors
on Amazon
Show steps
- Read chapters on data modeling, storage, and processing.
- Identify key concepts and architectures for data-intensive applications.
Create a Data Glossary
Show steps
Strengthen your data management skills by creating a data glossary for a specific domain or project.
Show steps
- Identify the relevant data terms and definitions.
- Organize the terms into a structured and logical framework.
- Publish the data glossary for reference and use.
Design a Data Lake
Show steps
Apply your knowledge of data storage and management by designing a data lake for a specific use case.
Browse courses on
Data Lake
Show steps
- Define the requirements and scope of your data lake.
- Choose appropriate data storage technologies and tools.
- Design the data lake architecture and data pipeline.
Career center
Reading list
Share
Similar courses
OpenCourser helps millions of learners each year. People visit us to learn workspace skills, ace their exams, and nurture their curiosity.
Our extensive catalog contains over 50,000 courses and twice as many books. Browse by search, by topic, or even by career interests. We'll match you to the right resources quickly.
Find this site helpful? Tell a friend about us.
We're supported by our community of learners. When you purchase or subscribe to courses and programs or purchase books, we may earn a commission from our partners.
Your purchases help us maintain our catalog and keep our servers humming without ads.
Thank you for supporting OpenCourser.
