We may earn an affiliate commission when you visit our partners.
Noah Gift
In this course, you will:
- Gain proficiency in using Linux for data engineering tasks
- Learn to configure and customize your Bash shell environment
- Understand how to manipulate data using shell variables and standard streams
- Construct powerful Bash scripts and command-line tools to automate workflows
- Master file and data management using Linux utilities and shell scripting
Read more
In this course, you will:
- Gain proficiency in using Linux for data engineering tasks
- Learn to configure and customize your Bash shell environment
- Understand how to manipulate data using shell variables and standard streams
- Construct powerful Bash scripts and command-line tools to automate workflows
- Master file and data management using Linux utilities and shell scripting
Read more
In this course, you will:
- Gain proficiency in using Linux for data engineering tasks
- Learn to configure and customize your Bash shell environment
- Understand how to manipulate data using shell variables and standard streams
- Construct powerful Bash scripts and command-line tools to automate workflows
- Master file and data management using Linux utilities and shell scripting
Whether you're a data engineer, software developer, or system administrator, this course will equip you with the Linux and Bash skills necessary to tackle real-world data challenges efficiently and effectively. Enhance your productivity and streamline your data workflows with the power of Linux and Bash scripting.
Register for this course and see more details by visiting:
OpenCourser.com/course/y6ozhr/linux
What's inside
Learning objectives
- Navigating and managing linux filesystems
- Configuring and customizing bash shell environments
- Manipulating data using shell variables and streams
- Building bash scripts and command-line tools
- Searching and processing text data in linux
- Automating data workflows with shell scripting
- Navigating and managing linux filesystems
- Configuring and customizing bash shell environments
- Manipulating data using shell variables and streams
- Building bash scripts and command-line tools
- Searching and processing text data in linux
- Automating data workflows with shell scripting
Syllabus
Module 1: Using Linux (18 hours)
\- Welcome to Linux and Bash for Data Engineering (Video, 0 minutes, Preview module)
\- Meet your Course Instructor: Noah Gift (Video, 0 minutes)
Read more
Syllabus
Module 1: Using Linux (18 hours)
\- Welcome to Linux and Bash for Data Engineering (Video, 0 minutes, Preview module)
\- Meet your Course Instructor: Noah Gift (Video, 0 minutes)
Read more
\- Overview of Key Concepts (Video, 4 minutes)
\- Introduction to Working with Linux (Video, 0 minutes)
\- Introduction to the Linux Shell (Video, 2 minutes)
\- Installing and Accessing Linux (Video, 11 minutes)
\- Getting Started with Github Codespaces (Video, 9 minutes)
\- Comparing Linux Shell Environments (Video, 9 minutes)
\- Writing 'Hello World' Bash Script (Video, 6 minutes)
\- Popular Terminal Commands in Linux (Video, 14 minutes)
\- Introduction to Shell Piping (Video, 1 minute)
\- What is Shell Piping? (Video, 3 minutes)
\- Walkthrough of Shell Pipeline Examples (Video, 12 minutes)
\- Introduction to SSH (Video, 1 minute)
\- What is SSH? (Video, 4 minutes)
\- Creating SSH Keys and using with GitHub (Video, 4 minutes)
\- Using SSH to Connect to Remote Server in AWS Cloud (Video, 7 minutes)
\- Walkthrough of SSH Tunneling Example (Video, 4 minutes)
\- Meet your Supporting Instructors: Kennedy Behrman and Alfredo Deza (Reading, 10 minutes)
\- Course Structure and Discussion Etiquette (Reading, 10 minutes)
\- Introduction to Working with Bash (Video, 0 minutes, Preview module)
\- Amazon CodeWhisperer CLI (Reading, 10 minutes)
\- Introduction to Configuring your Bash Shell Environment (Video, 2 minutes)
\- Shell Pipelines (Reading, 10 minutes)
\- Securely Connecting to a Linux Instance with SSH (Reading, 10 minutes)
\- Secure Remote Development with VSCode and SSH (Reading, 10 minutes)
\- Quiz-Configuring the Shell Environment (Assignment, 180 minutes)
\- Using Linux (Quiz, 30 minutes)
\- Quiz-Using Linux (Assignment, 180 minutes)
\- Quiz-How Shell Piping Works (Assignment, 180 minutes)
\- Quiz-Using SSH (Assignment, 180 minutes)
\- Meet and Greet (optional) (Discussion Prompt, 10 minutes)
\- Practicing Common Linux Shell Commands (Ungraded Lab, 60 minutes)
\- Using Linux Lab Challenge (Ungraded Lab, 60 minutes)
\- Practicing Common Shell Piping Operations (Ungraded Lab, 60 minutes)
\- Creating SSH Keys Locally (Ungraded Lab, 60 minutes)
\- Creating a Shell Pipeline (Ungraded Lab, 60 minutes)
Module 2: Using Bash (14 hours)
\- What are Shell Configuration Files? (Video, 5 minutes)
\- Configuring .bashrc (Video, 11 minutes)
\- Configuring .zshrc and Third-Party Tool oh-my-zsh (Video, 9 minutes)
\- Introduction to Working with Shell Variables (Video, 1 minute)
\- What are Shell Variables? (Video, 4 minutes)
\- Exporting Shell Variables in an Interactive Prompt (Video, 6 minutes)
\- Using Shell Variables in a Bash Script (Video, 7 minutes)
\- Introduction to Standard Streams (Video, 1 minute)
\- What are Standard Streams? (Video, 2 minutes)
\- Using Standard Out (Video, 5 minutes)
\- Using Standard In (Video, 2 minutes)
\- Using Standard Error (Video, 2 minutes)
\- Quiz-Working with Shell Variables (Assignment, 180 minutes)
\- Quiz-Standard Streams: Using Standard In, Standard Out, Standard Error (Assignment, 180 minutes)
\- Using Bash (Quiz, 30 minutes)
\- Configuring the Bash Shell (Ungraded Lab, 60 minutes)
\- Sourcing Shell Variables from a Script (Ungraded Lab, 60 minutes)
\- Practicing with Bash Streams (Ungraded Lab, 60 minutes)
Module 3: Building Bash Scripts (15 hours)
\- Introduction to Building Bash Scripts (Video, 0 minutes, Preview module)
\- Introduction to Shell Logic and Control Flow (Video, 0 minutes)
\- What is Shell Control Flow? (Video, 2 minutes)
\- Using Shell Control Flow to Solve Problems in Bash (Video, 2 minutes)
\- Using Shell Loops in Bash (Video, 6 minutes)
\- Evaluating Conditions in Bash (Video, 5 minutes)
\- Introduction to Manipulating Data in the Bash Shell (Video, 1 minute)
\- Shell Techniques for Data in the Bash Shell (Video, 2 minutes)
\- Truncating Data in Bash (Video, 3 minutes)
\- Filtering Data in Bash (Video, 3 minutes)
\- Searching Data in Bash (Video, 4 minutes)
\- Introduction to Writing Scripts and Command-Line Tools in Bash (Video, 0 minutes)
\- What are Scripts and Command-Line Tools? (Video, 3 minutes)
\- Building a Bash Script (Video, 4 minutes)
\- Building a Bash Function (Video, 5 minutes)
\- Building a Bash Command-Line Tool (Video, 3 minutes)
\- Bash-Adjacent Tools: Makefiles and Dockerfiles (Video, 7 minutes)
\- Data Structures in Bash (Video, 4 minutes)
\- Lesson Reflection (Reading, 10 minutes)
\- Key Terms (Reading, 10 minutes)
\- Final Course Challenge: Extended Phrase Repeater (Ungraded Lab, 60 minutes)
\- Building Bash Scripts (Quiz, 30 minutes)
\- Quiz-Using Shell Logic and Control Flow (Assignment, 180 minutes)
\- Quiz-Manipulating Data with the Shell (Assignment, 180 minutes)
\- Quiz-Writing Bash Scripts and Command-Line Tools (Assignment, 180 minutes)
\- Writing Loops in Bash (Ungraded Lab, 60 minutes)
\- Truncating Large Files with Bash (Ungraded Lab, 60 minutes)
\- Building a Command-Line Tool for Data Processing (Ungraded Lab, 60 minutes)
\- Building a Bash CLI with Options (Ungraded Lab, 60 minutes)
Module 4: Composing File and Data Solutions (16 hours)
\- Introduction to Composing File and Data Management Solutions in Linux (Video, 0 minutes, Preview module)
\- Introduction to Searching a Filesystem in Linux (Video, 0 minutes)
\- Methods for Searching a Filesystem in Linux (Video, 4 minutes)
\- Using the Locate Command to Find Files in Linux (Video, 3 minutes)
\- Using the Find Command to Find Files in Linux (Video, 2 minutes)
\- Using xargs to Extend Search Capabilities in Linux (Video, 2 minutes)
\- Using mdfind on OS X (Video, 6 minutes)
\- Introduction to Modifying Files, Directories, Permissions and Archiving in Linux (Video, 0 minutes)
\- Overview of Modifying the Filesystem in Linux (Video, 3 minutes)
\- Moving Files and Directories in Linux (Video, 6 minutes)
\- Setting Permissions on Files and Directories in Linux (Video, 4 minutes)
\- Archiving Data in Linux (Video, 3 minutes)
\- Introduction to Processing Text in Linux (Video, 0 minutes)
\- Benefits of Processing Text in Linux (Video, 1 minute)
\- Using grep, cut, sort and unique in Linux (Video, 8 minutes)
\- Editing with Truncation, awk and sed in Linux (Video, 7 minutes)
\- Using Regular Expressions (regex) in Linux (Video, 5 minutes)
\- Next Steps (Reading, 10 minutes)
\- Composing File and Data Management Solutions with Linux (Quiz, 30 minutes)
\- Quiz-Searching the Filesystem (Assignment, 180 minutes)
\- Quiz-Modifying Files, Directories, Permissions and Archiving (Assignment, 180 minutes)
\- Quiz-Processing Text (Assignment, 180 minutes)
\- Searching from the Bash Prompt (Ungraded Lab, 60 minutes)
\- Practicing the Basics of Shell Permissions (Ungraded Lab, 60 minutes)
\- Searching for Text with Bash (Ungraded Lab, 60 minutes)
\- Searching the filesystem with find (Ungraded Lab, 60 minutes)
Good to know
Good to know
Know what's good ,
what to watch for , and
possible dealbreakers
Develops Bash scripting competence in Linux, useful to system administrators, data engineers, and software developers
Builds a strong foundation of Linux fundamentals
Incorporates hands-on labs and interactive material for practical experience
Taught by experts in data engineering
Requires knowledge of Linux and command-line tools
Lacks focus on real-world projects
Save this course
Save Linux and Bash for Data Engineering to your list so you can find it easily later:
Save
Save
Activities
Be better prepared
before
your course. Deepen your understanding
during
and
after
it. Supplement your coursework and achieve mastery of the topics covered
in Linux and Bash for Data Engineering with these
activities:
Review Shell commands
Show steps
Refresh your memory on basic Shell commands to strengthen your foundation for this course.
Browse courses on
Shell Commands
Show steps
-
Review common commands for file and directory manipulation
-
Recall commands for searching and filtering data
-
Practice using commands for text processing and manipulation
Practice Bash shell commands
Show steps
Practicing Bash commands will help you gain proficiency in using them for various data engineering tasks.
Browse courses on
Bash
Show steps
-
Install a Linux operating system or use a virtual machine.
-
Open a terminal window.
-
Practice common commands such as ls, cd, mkdir, and rm.
-
Use Bash shell piping to combine commands.
-
Write simple Bash scripts to automate tasks.
Read 'Advanced Bash Scripting Guide'
Show steps
Delve into a comprehensive guide on Bash scripting to enhance your understanding of advanced concepts and techniques.
View
Advanced Bash Scripting Guide - Volume 1: An in...
on Amazon
Show steps
-
Read and study key chapters on advanced scripting topics
-
Apply the concepts and techniques to your own scripting projects
Nine other activities
Expand to see all activities and additional details
Show all 12 activities
Follow online tutorials on Bash scripting
Show steps
Following online tutorials will provide you with structured guidance and examples to enhance your Bash scripting skills.
Browse courses on
Bash
Show steps
-
Search for online tutorials on Bash scripting.
-
Choose a tutorial that aligns with your learning goals.
-
Follow the tutorial step-by-step.
-
Practice the concepts learned in the tutorial.
-
Explore additional tutorials to expand your knowledge.
Participate in a Bash scripting study group
Show steps
Participating in a study group will provide you with a supportive environment to learn and practice Bash scripting with others.
Browse courses on
Bash Scripting
Show steps
-
Find or create a Bash scripting study group.
-
Meet regularly with the group.
-
Discuss concepts and work on projects together.
-
Share knowledge and best practices.
-
Provide feedback and support to other members.
Practice Bash scripting
Show steps
Sharpen your Bash scripting skills through repetitive exercises to solidify your understanding of scripting concepts.
Browse courses on
Bash Scripting
Show steps
-
Create scripts to automate common tasks
-
Build scripts with control flow and error handling
-
Practice writing scripts to process and manipulate data
Attend a Bash scripting workshop
Show steps
Attending a workshop will provide you with hands-on experience and expert guidance in Bash scripting.
Browse courses on
Bash Scripting
Show steps
-
Research upcoming Bash scripting workshops.
-
Register for a workshop that meets your schedule and learning needs.
-
Attend the workshop and actively participate in the exercises.
-
Network with other participants and instructors.
-
Apply the knowledge and skills learned in the workshop to your own projects.
Join a study group
Show steps
Collaborate with peers in a study group for regular discussions, knowledge sharing, and support.
Show steps
-
Find or create a study group with fellow course participants
-
Establish regular meeting times and topics for discussion
-
Engage in active discussions and share insights
-
Provide support and motivation to group members
Build a data processing script using Bash
Show steps
Building a data processing script will allow you to apply your knowledge of Bash and data manipulation to solve real-world problems.
Browse courses on
Bash Scripting
Show steps
-
Identify a data processing task.
-
Design the Bash script using control flow, loops, and data manipulation techniques.
-
Implement the script.
-
Test the script with different data sets.
-
Refine the script based on testing results.
Explore advanced Linux utilities
Show steps
Expand your knowledge by exploring tutorials on advanced Linux utilities to enhance your data analysis capabilities.
Show steps
-
Find and follow tutorials on utilities like awk, sed, and grep
-
Apply these utilities to solve practical data manipulation challenges
Attend a Linux data engineering workshop
Show steps
Participate in a workshop focused on Linux data engineering to gain hands-on experience and learn from experts.
Show steps
-
Research and find a relevant workshop
-
Register and attend the workshop
-
Actively participate in exercises and discussions
Contribute to an open-source Bash project
Show steps
Enhance your knowledge and contribute to the community by participating in an open-source Bash project.
Show steps
-
Find an open-source Bash project that aligns with your interests
-
Review the project's documentation and codebase
-
Identify areas where you can contribute your skills
-
Submit your contributions and engage with the project community
Review Shell commands
Show steps
Refresh your memory on basic Shell commands to strengthen your foundation for this course.
Browse courses on
Shell Commands
Show steps
Show steps
Show steps
- Review common commands for file and directory manipulation
- Recall commands for searching and filtering data
- Practice using commands for text processing and manipulation
Practice Bash shell commands
Show steps
Practicing Bash commands will help you gain proficiency in using them for various data engineering tasks.
Browse courses on
Bash
Show steps
Show steps
Show steps
- Install a Linux operating system or use a virtual machine.
- Open a terminal window.
- Practice common commands such as ls, cd, mkdir, and rm.
- Use Bash shell piping to combine commands.
- Write simple Bash scripts to automate tasks.
Read 'Advanced Bash Scripting Guide'
Show steps
Delve into a comprehensive guide on Bash scripting to enhance your understanding of advanced concepts and techniques.
View
Advanced Bash Scripting Guide - Volume 1: An in...
on Amazon
Show steps
Show steps
Show steps
- Read and study key chapters on advanced scripting topics
- Apply the concepts and techniques to your own scripting projects
Nine other activities
Expand to see all activities and additional details
Show all 12 activities
Follow online tutorials on Bash scripting
Show steps
Following online tutorials will provide you with structured guidance and examples to enhance your Bash scripting skills.
Browse courses on
Bash
Show steps
Show steps
Show steps
- Search for online tutorials on Bash scripting.
- Choose a tutorial that aligns with your learning goals.
- Follow the tutorial step-by-step.
- Practice the concepts learned in the tutorial.
- Explore additional tutorials to expand your knowledge.
Participate in a Bash scripting study group
Show steps
Participating in a study group will provide you with a supportive environment to learn and practice Bash scripting with others.
Browse courses on
Bash Scripting
Show steps
Show steps
Show steps
- Find or create a Bash scripting study group.
- Meet regularly with the group.
- Discuss concepts and work on projects together.
- Share knowledge and best practices.
- Provide feedback and support to other members.
Practice Bash scripting
Show steps
Sharpen your Bash scripting skills through repetitive exercises to solidify your understanding of scripting concepts.
Browse courses on
Bash Scripting
Show steps
Show steps
Show steps
- Create scripts to automate common tasks
- Build scripts with control flow and error handling
- Practice writing scripts to process and manipulate data
Attend a Bash scripting workshop
Show steps
Attending a workshop will provide you with hands-on experience and expert guidance in Bash scripting.
Browse courses on
Bash Scripting
Show steps
Show steps
Show steps
- Research upcoming Bash scripting workshops.
- Register for a workshop that meets your schedule and learning needs.
- Attend the workshop and actively participate in the exercises.
- Network with other participants and instructors.
- Apply the knowledge and skills learned in the workshop to your own projects.
Join a study group
Show steps
Collaborate with peers in a study group for regular discussions, knowledge sharing, and support.
Show steps
Show steps
Show steps
- Find or create a study group with fellow course participants
- Establish regular meeting times and topics for discussion
- Engage in active discussions and share insights
- Provide support and motivation to group members
Build a data processing script using Bash
Show steps
Building a data processing script will allow you to apply your knowledge of Bash and data manipulation to solve real-world problems.
Browse courses on
Bash Scripting
Show steps
Show steps
Show steps
- Identify a data processing task.
- Design the Bash script using control flow, loops, and data manipulation techniques.
- Implement the script.
- Test the script with different data sets.
- Refine the script based on testing results.
Explore advanced Linux utilities
Show steps
Expand your knowledge by exploring tutorials on advanced Linux utilities to enhance your data analysis capabilities.
Show steps
Show steps
Show steps
- Find and follow tutorials on utilities like awk, sed, and grep
- Apply these utilities to solve practical data manipulation challenges
Attend a Linux data engineering workshop
Show steps
Participate in a workshop focused on Linux data engineering to gain hands-on experience and learn from experts.
Show steps
Show steps
Show steps
- Research and find a relevant workshop
- Register and attend the workshop
- Actively participate in exercises and discussions
Contribute to an open-source Bash project
Show steps
Enhance your knowledge and contribute to the community by participating in an open-source Bash project.
Show steps
Show steps
Show steps
- Find an open-source Bash project that aligns with your interests
- Review the project's documentation and codebase
- Identify areas where you can contribute your skills
- Submit your contributions and engage with the project community
Career center
Learners who complete Linux and Bash for Data Engineering will develop knowledge and skills
that may be useful to these careers:
For more career information including salaries, visit:
OpenCourser.com/course/y6ozhr/linux
Reading list
We haven't picked any books for this reading list yet.
For more information about how these books relate to this course, visit:
OpenCourser.com/course/y6ozhr/linux
Share
Similar courses
Here are nine courses similar to
Linux and Bash for Data Engineering.
Bash Shell Scripting Tutorial for Beginners
OpenCourser.com/course/l5mral/bash
Bash Shell Scripting Tutorial for Beginners
Most relevant
The Bash Shell and Basic Scripting in Linux
OpenCourser.com/course/trxtpr/the
The Bash Shell and Basic Scripting in Linux
Most relevant
Linux Commands & Shell Scripting
OpenCourser.com/course/i1okpq/linux
Linux Commands & Shell Scripting
Most relevant
Scripting for Security with Bash
OpenCourser.com/course/gdrrry/scripting
Scripting for Security with Bash
Most relevant
Getting Started with Shell Scripting for Bash and Z Shell
OpenCourser.com/course/6tnv28/getting
Getting Started with Shell Scripting for Bash and Z Shell
Most relevant
Bash Scripting and Shell Programming (Linux Command Line)
OpenCourser.com/course/ppckuj/bash
Bash Scripting and Shell Programming (Linux Command Line)
Most relevant
Creating Shell Scripts in Enterprise Linux
OpenCourser.com/course/0undos/creating
Creating Shell Scripts in Enterprise Linux
Most relevant
Using Python to Interact with the Operating System
OpenCourser.com/course/4cmlll/using
Using Python to Interact with the Operating System
Most relevant
Bash Shell Scripting: From Zero To Automation
OpenCourser.com/course/4qzuzt/bash
Bash Shell Scripting: From Zero To Automation
Most relevant
Effort
3 to 6 hours per week for 4 weeks
Level
Introductory
Via
edX
Institution
Pragmatic AI Labs
Instructor
Noah Gift
Session
March 31, 2024
- March 30, 2025
Language
English
Transcript
English
Good to know
Develops Bash scripting competence in Linux, useful to system administrators, data engineers, and software developers
Builds a strong foundation of Linux fundamentals
Incorporates hands-on labs and interactive material for practical experience
Taught by experts in data engineering
Requires knowledge of Linux and command-line tools
Lacks focus on real-world projects
Our mission
OpenCourser helps millions of learners each year. People visit us to learn workspace skills, ace their exams, and nurture their curiosity.
Our extensive catalog contains over 50,000 courses and twice as many books. Browse by search, by topic, or even by career interests. We'll match you to the right resources quickly.
Find this site helpful? Tell a friend about us.
Affiliate disclosure
We're supported by our community of learners. When you purchase or subscribe to courses and programs or purchase books, we may earn a commission from our partners.
Your purchases help us maintain our catalog and keep our servers humming without ads.
Thank you for supporting OpenCourser.
© 2016 - 2024 OpenCourser