The Building Blocks of Hadoop - HDFS, MapReduce, and YARN from Pluralsight

Processing billions of records requires a deep understanding of distributed computing. In this course, you'll get introduced to Hadoop, an open-source distributed computing framework that can help you do just that.

You know how to write Java code and you know what processing you want to perform on your huge dataset. But, can you use the Hadoop distributed framework effectively to get your work done?

This course,

, gives you a fundamental understanding of the building blocks of Hadoop:

HDFS is the Hadoop Distributed File System, the primary data storage system used by Hadoop applications to scale a single Apache Hadoop cluster to hundreds of nodes.

MapReduce is a framework and java-based programming model used for processing large amounts of data. The map procedure filters and sorts the data, and the reduce method performs a summary operation.

YARN stands for Yet Another Resource Negotiator. It is a large-scale, distributed operating system for big data apps that allows the data in HDFS to be processed and run by data processing engines.

This course will introduce you to Hadoop and its basic building blocks. Topics covered include:

Anyone who wants to learn Hadoop and its building blocks of HDFS, MapReduce, and YARN should take this tutorial! If you need help processing vast numbers of records and want to understand distributed computing, this course is for you.

If you know how to write Java code and you know what processing you want to perform on your huge dataset then you should be good to go in this course. No prior experience with Hadoop is required.

What's inside

Syllabus

Course Overview

Introducing Hadoop

Installing Hadoop

Storing Data with HDFS

Processing Data with MapReduce

Scheduling and Managing Tasks with YARN

Good to know

Know what's good

, what to watch for

, and possible dealbreakers

Teaches foundational skills for working with Hadoop

Covers storing data with HDFS, processing data with MapReduce, and managing tasks with YARN

Taught by Janani Ravi, who is recognized for their work in data science and distributed computing

Suitable for anyone who wants to learn Hadoop its building blocks, this tutorial is a solid introduction

Course assumes learners know how to write Java code and understand the processing they want to perform on their dataset

For learners who seek a deep dive into Hadoop and its applications, this course provides a good starting point

Activities

Be better prepared before your course. Deepen your understanding during and after it. Supplement your coursework and achieve mastery of the topics covered in The Building Blocks of Hadoop - HDFS, MapReduce, and YARN with these activities:

Review Java Programming

Show steps

Refreshes and strengthens Java skills for effective Hadoop development.

Browse courses on Java Programming

Show steps

Review Java syntax and coding best practices.
Practice writing Java programs.
Solve Java coding challenges.

Review Hadoop: The Definitive Guide

Show steps

Provides a deep understanding of core Hadoop concepts, preparing you for course material.

View Hadoop: The Definitive Guide: Storage and... on Amazon

Show steps

Obtain the book from your preferred source.
Identify the chapters that align with the course syllabus.
Read the selected chapters thoroughly.
Take notes on key concepts and technical details.
Review your notes regularly to reinforce your understanding.

Hadoop Study Group

Show steps

Fosters collaboration and knowledge sharing among learners.

Show steps

Form a study group with fellow learners.
Meet regularly to discuss course topics.
Share notes and resources.
Work on assignments or projects together.

Two other activities

Expand to see all activities and additional details

Show all five activities

Hadoop Best Practices Resource List

Show steps

Compiles and organizes valuable references for ongoing Hadoop development.

Show steps

Gather articles, tutorials, and documentation related to Hadoop best practices.
Create a structured list or document that organizes these resources.
Review and update the resource list regularly.

Hadoop Hackathon

Show steps

Provides a challenging environment to test skills and fuel innovation.

Show steps

Find or organize a Hadoop hackathon.
Form a team or participate individually.
Develop and implement a Hadoop-based solution to a problem.
Present your solution to a panel of judges.

Career center

Learners who complete The Building Blocks of Hadoop - HDFS, MapReduce, and YARN will develop knowledge and skills that may be useful to these careers:

Data Engineer

Data Engineers design, build, and maintain data systems. This course can help you build a foundation in Hadoop, which is a key technology used for processing large amounts of data. By understanding the building blocks of Hadoop, you can learn how to use it to design and build scalable data systems.

See salaries and explore the career path for Data Engineer

Data Scientist

Data Scientists use data to solve business problems. This course can help you build a foundation in Hadoop, which is a key technology used for processing large amounts of data. By understanding the building blocks of Hadoop, you can learn how to use it to solve business problems using data.

See salaries and explore the career path for Data Scientist

Business Analyst

Business Analysts use data to help businesses make informed decisions. This course can help you build a foundation in Hadoop, which is a key technology used for processing large amounts of data. By understanding the building blocks of Hadoop, you can learn how to use it to analyze data and solve business problems.

See salaries and explore the career path for Business Analyst

Cloud Engineer

Cloud Engineers design, build, and maintain cloud computing systems. This course can help you build a foundation in Hadoop, which is a key technology used for processing large amounts of data. By understanding the building blocks of Hadoop, you can learn how to use it to design and build cloud computing systems that can handle large amounts of data.

See salaries and explore the career path for Cloud Engineer

Machine Learning Engineer

Machine Learning Engineers design, build, and maintain machine learning models. This course can help you build a foundation in Hadoop, which is a key technology used for processing large amounts of data. By understanding the building blocks of Hadoop, you can learn how to use it to develop machine learning models that can handle large amounts of data.

See salaries and explore the career path for Machine Learning Engineer

Data Analyst

Data Analysts collect, clean, and analyze data to help businesses make informed decisions. This course can help you build a foundation in Hadoop, which is a key technology used for processing large amounts of data. By understanding the building blocks of Hadoop, you can learn how to use it to effectively analyze data and solve business problems.

See salaries and explore the career path for Data Analyst

Product Manager

Product Managers manage the development and launch of new products. This course can help you build a foundation in Hadoop, which is a key technology used for processing large amounts of data. By understanding the building blocks of Hadoop, you can learn how to use it to develop and launch products that can handle large amounts of data.

See salaries and explore the career path for Product Manager

System Administrator

System Administrators manage and maintain computer systems. This course can help you build a foundation in Hadoop, which is a key technology used for processing large amounts of data. By understanding the building blocks of Hadoop, you can learn how to use it to manage and maintain computer systems that can handle large amounts of data.

See salaries and explore the career path for System Administrator

Software Engineer

Software Engineers design, build, and maintain software applications. This course can help you build a foundation in Hadoop, which is a key technology used for processing large amounts of data. By understanding the building blocks of Hadoop, you can learn how to use it to develop software applications that can handle large amounts of data.

See salaries and explore the career path for Software Engineer

Database Administrator

Database Administrators manage and maintain databases. This course can help you build a foundation in Hadoop, which is a key technology used for processing large amounts of data. By understanding the building blocks of Hadoop, you can learn how to use it to manage and maintain large databases.

See salaries and explore the career path for Database Administrator

Project Manager

Project Managers plan and manage projects. This course can help you build a foundation in Hadoop, which is a key technology used for processing large amounts of data. By understanding the building blocks of Hadoop, you can learn how to use it to plan and manage projects that involve large amounts of data.

See salaries and explore the career path for Project Manager

Sales Engineer

Sales Engineers help customers understand and use technical products. This course can help you build a foundation in Hadoop, which is a key technology used for processing large amounts of data. By understanding the building blocks of Hadoop, you can learn how to help customers use Hadoop to solve their business problems.

See salaries and explore the career path for Sales Engineer

Technical Writer

Technical Writers create documentation for technical products. This course can help you build a foundation in Hadoop, which is a key technology used for processing large amounts of data. By understanding the building blocks of Hadoop, you can learn how to write documentation for products that use Hadoop.

See salaries and explore the career path for Technical Writer

Marketing Manager

Marketing Managers plan and execute marketing campaigns. This course can help you build a foundation in Hadoop, which is a key technology used for processing large amounts of data. By understanding the building blocks of Hadoop, you can learn how to use it to plan and execute marketing campaigns that reach large audiences.

See salaries and explore the career path for Marketing Manager

Operations Manager

Operations Managers plan and manage the day-to-day operations of a business. This course can help you build a foundation in Hadoop, which is a key technology used for processing large amounts of data. By understanding the building blocks of Hadoop, you can learn how to use it to plan and manage the operations of a business that uses Hadoop.

See salaries and explore the career path for Operations Manager