OpenCourser brand icon
Sign in
We may earn an affiliate commission when you visit our partners.
Course image
Course image
edX logo

Linux and Bash for Data Engineering

Noah Gift

In this course, you will:

Read more

In this course, you will:

  • Gain proficiency in using Linux for data engineering tasks
  • Learn to configure and customize your Bash shell environment
  • Understand how to manipulate data using shell variables and standard streams
  • Construct powerful Bash scripts and command-line tools to automate workflows
  • Master file and data management using Linux utilities and shell scripting

Whether you're a data engineer, software developer, or system administrator, this course will equip you with the Linux and Bash skills necessary to tackle real-world data challenges efficiently and effectively. Enhance your productivity and streamline your data workflows with the power of Linux and Bash scripting.

Enroll now

What's inside

Learning objectives

  • Navigating and managing linux filesystems
  • Configuring and customizing bash shell environments
  • Manipulating data using shell variables and streams
  • Building bash scripts and command-line tools
  • Searching and processing text data in linux
  • Automating data workflows with shell scripting

Syllabus

Module 1: Using Linux (18 hours)
\- Welcome to Linux and Bash for Data Engineering (Video, 0 minutes, Preview module)
\- Meet your Course Instructor: Noah Gift (Video, 0 minutes)
Read more
\- Overview of Key Concepts (Video, 4 minutes)
\- Introduction to Working with Linux (Video, 0 minutes)
\- Introduction to the Linux Shell (Video, 2 minutes)
\- Installing and Accessing Linux (Video, 11 minutes)
\- Getting Started with Github Codespaces (Video, 9 minutes)
\- Comparing Linux Shell Environments (Video, 9 minutes)
\- Writing 'Hello World' Bash Script (Video, 6 minutes)
\- Popular Terminal Commands in Linux (Video, 14 minutes)
\- Introduction to Shell Piping (Video, 1 minute)
\- What is Shell Piping? (Video, 3 minutes)
\- Walkthrough of Shell Pipeline Examples (Video, 12 minutes)
\- Introduction to SSH (Video, 1 minute)
\- What is SSH? (Video, 4 minutes)
\- Creating SSH Keys and using with GitHub (Video, 4 minutes)
\- Using SSH to Connect to Remote Server in AWS Cloud (Video, 7 minutes)
\- Walkthrough of SSH Tunneling Example (Video, 4 minutes)
\- Meet your Supporting Instructors: Kennedy Behrman and Alfredo Deza (Reading, 10 minutes)
\- Course Structure and Discussion Etiquette (Reading, 10 minutes)
\- Introduction to Working with Bash (Video, 0 minutes, Preview module)
\- Amazon CodeWhisperer CLI (Reading, 10 minutes)
\- Introduction to Configuring your Bash Shell Environment (Video, 2 minutes)
\- Shell Pipelines (Reading, 10 minutes)
\- Securely Connecting to a Linux Instance with SSH (Reading, 10 minutes)
\- Secure Remote Development with VSCode and SSH (Reading, 10 minutes)
\- Quiz-Configuring the Shell Environment (Assignment, 180 minutes)
\- Using Linux (Quiz, 30 minutes)
\- Quiz-Using Linux (Assignment, 180 minutes)
\- Quiz-How Shell Piping Works (Assignment, 180 minutes)
\- Quiz-Using SSH (Assignment, 180 minutes)
\- Meet and Greet (optional) (Discussion Prompt, 10 minutes)
\- Practicing Common Linux Shell Commands (Ungraded Lab, 60 minutes)
\- Using Linux Lab Challenge (Ungraded Lab, 60 minutes)
\- Practicing Common Shell Piping Operations (Ungraded Lab, 60 minutes)
\- Creating SSH Keys Locally (Ungraded Lab, 60 minutes)
\- Creating a Shell Pipeline (Ungraded Lab, 60 minutes)
Module 2: Using Bash (14 hours)
\- What are Shell Configuration Files? (Video, 5 minutes)
\- Configuring .bashrc (Video, 11 minutes)
\- Configuring .zshrc and Third-Party Tool oh-my-zsh (Video, 9 minutes)
\- Introduction to Working with Shell Variables (Video, 1 minute)
\- What are Shell Variables? (Video, 4 minutes)
\- Exporting Shell Variables in an Interactive Prompt (Video, 6 minutes)
\- Using Shell Variables in a Bash Script (Video, 7 minutes)
\- Introduction to Standard Streams (Video, 1 minute)
\- What are Standard Streams? (Video, 2 minutes)
\- Using Standard Out (Video, 5 minutes)
\- Using Standard In (Video, 2 minutes)
\- Using Standard Error (Video, 2 minutes)
\- Quiz-Working with Shell Variables (Assignment, 180 minutes)
\- Quiz-Standard Streams: Using Standard In, Standard Out, Standard Error (Assignment, 180 minutes)
\- Using Bash (Quiz, 30 minutes)
\- Configuring the Bash Shell (Ungraded Lab, 60 minutes)
\- Sourcing Shell Variables from a Script (Ungraded Lab, 60 minutes)
\- Practicing with Bash Streams (Ungraded Lab, 60 minutes)
Module 3: Building Bash Scripts (15 hours)
\- Introduction to Building Bash Scripts (Video, 0 minutes, Preview module)
\- Introduction to Shell Logic and Control Flow (Video, 0 minutes)
\- What is Shell Control Flow? (Video, 2 minutes)
\- Using Shell Control Flow to Solve Problems in Bash (Video, 2 minutes)
\- Using Shell Loops in Bash (Video, 6 minutes)
\- Evaluating Conditions in Bash (Video, 5 minutes)
\- Introduction to Manipulating Data in the Bash Shell (Video, 1 minute)
\- Shell Techniques for Data in the Bash Shell (Video, 2 minutes)
\- Truncating Data in Bash (Video, 3 minutes)
\- Filtering Data in Bash (Video, 3 minutes)
\- Searching Data in Bash (Video, 4 minutes)
\- Introduction to Writing Scripts and Command-Line Tools in Bash (Video, 0 minutes)
\- What are Scripts and Command-Line Tools? (Video, 3 minutes)
\- Building a Bash Script (Video, 4 minutes)
\- Building a Bash Function (Video, 5 minutes)
\- Building a Bash Command-Line Tool (Video, 3 minutes)
\- Bash-Adjacent Tools: Makefiles and Dockerfiles (Video, 7 minutes)
\- Data Structures in Bash (Video, 4 minutes)
\- Lesson Reflection (Reading, 10 minutes)
\- Key Terms (Reading, 10 minutes)
\- Final Course Challenge: Extended Phrase Repeater (Ungraded Lab, 60 minutes)
\- Building Bash Scripts (Quiz, 30 minutes)
\- Quiz-Using Shell Logic and Control Flow (Assignment, 180 minutes)
\- Quiz-Manipulating Data with the Shell (Assignment, 180 minutes)
\- Quiz-Writing Bash Scripts and Command-Line Tools (Assignment, 180 minutes)
\- Writing Loops in Bash (Ungraded Lab, 60 minutes)
\- Truncating Large Files with Bash (Ungraded Lab, 60 minutes)
\- Building a Command-Line Tool for Data Processing (Ungraded Lab, 60 minutes)
\- Building a Bash CLI with Options (Ungraded Lab, 60 minutes)
Module 4: Composing File and Data Solutions (16 hours)
\- Introduction to Composing File and Data Management Solutions in Linux (Video, 0 minutes, Preview module)
\- Introduction to Searching a Filesystem in Linux (Video, 0 minutes)
\- Methods for Searching a Filesystem in Linux (Video, 4 minutes)
\- Using the Locate Command to Find Files in Linux (Video, 3 minutes)
\- Using the Find Command to Find Files in Linux (Video, 2 minutes)
\- Using xargs to Extend Search Capabilities in Linux (Video, 2 minutes)
\- Using mdfind on OS X (Video, 6 minutes)
\- Introduction to Modifying Files, Directories, Permissions and Archiving in Linux (Video, 0 minutes)
\- Overview of Modifying the Filesystem in Linux (Video, 3 minutes)
\- Moving Files and Directories in Linux (Video, 6 minutes)
\- Setting Permissions on Files and Directories in Linux (Video, 4 minutes)
\- Archiving Data in Linux (Video, 3 minutes)
\- Introduction to Processing Text in Linux (Video, 0 minutes)
\- Benefits of Processing Text in Linux (Video, 1 minute)
\- Using grep, cut, sort and unique in Linux (Video, 8 minutes)
\- Editing with Truncation, awk and sed in Linux (Video, 7 minutes)
\- Using Regular Expressions (regex) in Linux (Video, 5 minutes)
\- Next Steps (Reading, 10 minutes)
\- Composing File and Data Management Solutions with Linux (Quiz, 30 minutes)
\- Quiz-Searching the Filesystem (Assignment, 180 minutes)
\- Quiz-Modifying Files, Directories, Permissions and Archiving (Assignment, 180 minutes)
\- Quiz-Processing Text (Assignment, 180 minutes)
\- Searching from the Bash Prompt (Ungraded Lab, 60 minutes)
\- Practicing the Basics of Shell Permissions (Ungraded Lab, 60 minutes)
\- Searching for Text with Bash (Ungraded Lab, 60 minutes)
\- Searching the filesystem with find (Ungraded Lab, 60 minutes)

Good to know

Know what's good
, what to watch for
, and possible dealbreakers
Develops Bash scripting competence in Linux, useful to system administrators, data engineers, and software developers
Builds a strong foundation of Linux fundamentals
Incorporates hands-on labs and interactive material for practical experience
Taught by experts in data engineering
Requires knowledge of Linux and command-line tools
Lacks focus on real-world projects

Save this course

Save Linux and Bash for Data Engineering to your list so you can find it easily later:
Save

Activities

Coming soon We're preparing activities for Linux and Bash for Data Engineering. These are activities you can do either before, during, or after a course.

Career center

Learners who complete Linux and Bash for Data Engineering will develop knowledge and skills that may be useful to these careers:

Reading list

We haven't picked any books for this reading list yet.

Share

Help others find this course page by sharing it with your friends and followers:
Facebook
LinkedIn
Reddit
X
Link
Email

Similar courses

Here are nine courses similar to Linux and Bash for Data Engineering.
Bash Shell Scripting Tutorial for Beginners
Most relevant
The Bash Shell and Basic Scripting in Linux
Most relevant
Linux Commands & Shell Scripting
Most relevant
Scripting for Security with Bash
Most relevant
Getting Started with Shell Scripting for Bash and Z Shell
Most relevant
Bash Scripting and Shell Programming (Linux Command Line)
Most relevant
Creating Shell Scripts in Enterprise Linux
Most relevant
Using Python to Interact with the Operating System
Most relevant
Bash Shell Scripting: From Zero To Automation
Most relevant
Course image
Effort
3 to 6 hours per week for 4 weeks
Level
Introductory
Via
edX
Institution
Pragmatic AI Labs
Instructor
Noah Gift
Session
March 31, 2024 - March 30, 2025
Language
English
Transcript
English
Good to know
Develops Bash scripting competence in Linux, useful to system administrators, data engineers, and software developers
Builds a strong foundation of Linux fundamentals
Incorporates hands-on labs and interactive material for practical experience
Taught by experts in data engineering
Requires knowledge of Linux and command-line tools
Lacks focus on real-world projects
Share and help others discover this course
Facebook LinkedIn X Reddit Link Email
Our mission

OpenCourser helps millions of learners each year. People visit us to learn workspace skills, ace their exams, and nurture their curiosity.

Our extensive catalog contains over 50,000 courses and twice as many books. Browse by search, by topic, or even by career interests. We'll match you to the right resources quickly.

Find this site helpful? Tell a friend about us.

Affiliate disclosure

We're supported by our community of learners. When you purchase or subscribe to courses and programs or purchase books, we may earn a commission from our partners.

Your purchases help us maintain our catalog and keep our servers humming without ads.

Thank you for supporting OpenCourser.

© 2016 - 2024 OpenCourser