We may earn an affiliate commission when you visit our partners.
Alfredo Deza
In this course, part of the Python, Bash and SQL Essentials for Data Engineering Specialization, you will gain the skills to work effectively with data using Python and SQL. You will learn to:
Read more
In this course, part of the Python, Bash and SQL Essentials for Data Engineering Specialization, you will gain the skills to work effectively with data using Python and SQL. You will learn to:
Read more
In this course, part of the Python, Bash and SQL Essentials for Data Engineering Specialization, you will gain the skills to work effectively with data using Python and SQL. You will learn to:
- Leverage Python's powerful data structures to load, manipulate, and analyze data
- Create robust Python scripts to automate data processing tasks
- Utilize SQLite to store and retrieve data from within your Python scripts
- Extract data from websites using web scraping techniques and persist it in databases
- Work with MySQL databases using modern tools like VSCode to execute queries and manage data
Whether you are a data engineer, analyst, or aspiring data professional, this course will equip you with the essential skills to efficiently handle data and drive meaningful insights in today's data-driven world.
Register for this course and see more details by visiting:
OpenCourser.com/course/iw9sje/scripting
What's inside
Learning objectives
- Manipulate data using python's built-in data structures
- Create python scripts to automate data tasks
- Use sqlite to store and query data in python
- Extract data from websites via web scraping
- Execute mysql queries and operations in vscode
- Import and export data in mysql databases
- Manipulate data using python's built-in data structures
- Create python scripts to automate data tasks
- Use sqlite to store and query data in python
- Extract data from websites via web scraping
- Execute mysql queries and operations in vscode
- Import and export data in mysql databases
Syllabus
Here is the course structure formatted with bullets for each module:
Module 1: Working with Data in Python (5 hours)
\- Videos (Total 62 minutes):
Read more
Syllabus
Here is the course structure formatted with bullets for each module:
Module 1: Working with Data in Python (5 hours)
\- Videos (Total 62 minutes):
Read more
\- Welcome to Scripting with Python and SQL for Data Engineering (Preview module)
\- Meet your Course Instructor: Alfredo Deza (0 minutes)
\- Overview of Key Concepts (4 minutes)
\- Introduction to Working with Data in Python (0 minutes)
\- Introduction to Python Data Structures (0 minutes)
\- Using Lists to Save and Retrieve Data in Python (5 minutes)
\- Using Dictionaries to Save and Retrieve Data in Python (7 minutes)
\- Overview of Less Common Data Structures in Python (4 minutes)
\- Recap of Data Structures in Python (1 minute)
\- Introduction to Choosing Data Structures in Python (1 minute)
\- Iterating Over Lists and Dictionaries in Python (7 minutes)
\- Iterating Over Other Data Structures in Python (2 minutes)
\- Storing Data Between Data Structures in Python (6 minutes)
\- Recap of Mapping Data Structures in Python (1 minute)
\- Introduction to Data Sources and Formats in Python (0 minutes)
\- Loading Data from Files and File Paths in Python (6 minutes)
\- Working with JSON in Python (6 minutes)
\- Saving Data from Python to Disk (4 minutes)
\- Recap of Persisting and Loading Data in Python (0 minutes)
\- Readings (Total 80 minutes):
\- Connect with your instructor (10 minutes)
\- Meet your Supporting Instructors: Kennedy Behrman and Noah Gift (10 minutes)
\- Course Structure and Discussion Etiquette (10 minutes)
\- Key Terms (10 minutes)
\- Python Lists and Dictionaries (10 minutes)
\- Build a notebook that reformats data into JSON (10 minutes)
\- Quizzes (Total 60 minutes):
\- Using Data Structures in Python (30 minutes)
\- Working with Data in Python (30 minutes)
\- Discussion Prompt (Total 10 minutes):
\- Meet and Greet (optional) (10 minutes)
\- Ungraded Labs (Total 120 minutes):
\- Videos (Total 69 minutes):
\- Looping Over Lists and Dictionaries in Python (60 minutes)
\- Loading and Saving JSON Files (60 minutes)
Module 2: Python Scripting and SQL (5 hours)
\- Videos (Total 56 minutes):
\- Introduction to Python Scripting and SQL (Preview module)
\- Introduction to Scripting in Python (0 minutes)
\- Creating a Script as a Module in Python (5 minutes)
\- Traversing the File System with a Script in Python (7 minutes)
\- Recap of Python Scripting Basics (0 minutes)
\- Introduction to Embedded Databases (0 minutes)
\- What is SQLite? (2 minutes)
\- Creating and Connecting to a SQLite Database in Python (5 minutes)
\- Saving and Querying from a SQLite Database in Python (10 minutes)
\- Recap of SQLite and Python (1 minute)
\- Introduction to Querying with SQL in Python (1 minute)
\- Basic SQL Commands in Python (6 minutes)
\- Extracting Distinct Data using SQL in Python (7 minutes)
\- Searching with SQL in Python (5 minutes)
\- Recap of Querying with SQL in Python (1 minute)
\- Readings (Total 70 minutes):
\- Create a Reporting Script for File Sizes (10 minutes)
\- Minimal Python, Chapter 2: Learn to Store Data (10 minutes)
\- Recap of Persistence and Efficacy with Web Scraping (1 minute)
\- Example GitHub Repository (10 minutes)
\- Quiz (Total 30 minutes):
\- Introduction to Working with MySQL (Preview module)
\- Python Scripting and SQL (30 minutes)
\- Ungraded Labs (Total 180 minutes):
\- Build a Script to Find Large Files (60 minutes)
\- Create a SQLite Database and Store Data (60 minutes)
\- Querying a SQLite Database (60 minutes)
Module 3: Web Scraping using Python (4 hours)
\- Videos (Total 64 minutes):
\- Introduction to Web Scraping using Python (Preview module)
\- Introduction to Extracting Data from Unstructured HTML (1 minute)
\- Challenges with Web Data (3 minutes)
\- Parsing HTML with HTMLParser in Python (5 minutes)
\- Recap of Web Scraping Techniques in Python (0 minutes)
\- Introduction to Scrapy and XPath in Python (1 minute)
\- Creating a Web Scraping Project with Scrapy in Python (6 minutes)
\- Parsing Data with XPath and Scrapy Shell (11 minutes)
\- Using Scrapy Spider for Web Scraping (5 minutes)
\- Recap of Scrapy and XPath in Python (0 minutes)
\- Overview of Challenges with Web Scraping (2 minutes)
\- Scraping Locally (8 minutes)
\- Persisting Data in CSV and JSON Formats (9 minutes)
\- Persisting Data to a SQLite Database (6 minutes)
\- Parsing Techniques with HTMLParser (10 minutes)
\- Introduction to VSCode with MySQL (1 minute)
\- Codespaces Lab: Parse HTML with Scrapy and XPath (10 minutes)
\- Efficient Scraping Techniques (10 minutes)
\- Build a Web Scraping Tool (10 minutes)
\- Web Scraping using Python (30 minutes)
\- Parse HTML with HTMLParser (60 minutes)
\- Parse HTML and Persist it to a SQLite Database (60 minutes)
Module 4: Working with MySQL (7 hours)
\- Connecting to a MySQL Server (8 minutes)
\- Recap of VSCode with MySQL (1 minute)
\- Challenges of Running MySQL Queries (1 minute)
\- Using VSCode to Execute MySQL Queries (6 minutes)
\- Recap of Running MySQL Queries (1 minute)
\- Challenges with Importing to Databases (1 minute)
\- Importing CSV Data into MySQL (6 minutes)
\- Exporting Data from MySQL (6 minutes)
\- Recap of Importing and Exporting Data in MySQL (1 minute)
\- MySQL Hacking Overview (2 minutes)
\- MySQL from Terminal (3 minutes)
\- Archive and Drop Database (5 minutes)
\- Import external database Sakila (7 minutes)
\- Modify database Sakila (4 minutes)
\- Bash pipelines with MySQL (5 minutes)
\- MySQL to Python Standard Library Web Server (4 minutes)
\- Key terms (10 minutes)
\- Working with VSCode and MySQL (10 minutes)
\- Lesson Reflection (10 minutes)
\- Loading and Exporting Data using MySQL (10 minutes)
\- Next Steps (10 minutes)
\- Working with MySQL (30 minutes)
\- Ungraded Labs (Total 300 minutes):
\- Connect to a Running MySQL Server (60 minutes)
\- Running Queries Repository (60 minutes)
\- Export and Import Data in MySQL (60 minutes)
\- Linux Hacking with MySQL (60 minutes)
\- MySQL Sandbox (60 minutes)
Good to know
Good to know
Know what's good ,
what to watch for , and
possible dealbreakers
Designed for beginners, this course lays the foundational knowledge and core skills for those seeking to enter the field of data engineering
Taught by experienced professionals in data engineering, this course conveys real-world applications and practical techniques
Delves into the practical applications of Python, Bash, and SQL, equipping learners with industry-standard tools for data analysis and engineering
May require supplemental learning for those with limited programming experience, as it assumes some prior knowledge
Save this course
Save Scripting with Python and SQL for Data Engineering to your list so you can find it easily later:
Save
Save
Activities
Be better prepared
before
your course. Deepen your understanding
during
and
after
it. Supplement your coursework and achieve mastery of the topics covered
in Scripting with Python and SQL for Data Engineering with these
activities:
Join a study group or online forum for Python and data engineering discussions
Show steps
Engage with peers and experts in the field by actively participating in discussions, sharing knowledge, and collaborating on projects.
Browse courses on
Python
Show steps
-
Identify and join relevant study groups or online forums
-
Participate in discussions, ask questions, and share your insights
-
Collaborate on projects or group challenges
Review basic Python skills
Show steps
Let's brush up on the fundamentals of Python programming to ensure you have the necessary foundational skills for this course.
Browse courses on
Python
Show steps
-
Review variables and data types
-
Practice writing and executing simple Python scripts
-
Create a few basic functions and classes
Complete an online Python tutorial
Show steps
Deepen your understanding of Python by following a guided tutorial that covers data structures and their applications.
Browse courses on
Python
Show steps
-
Identify a reputable online tutorial or course
-
Follow the tutorial step-by-step, practicing the concepts
-
Complete any exercises or assignments provided in the tutorial
Five other activities
Expand to see all activities and additional details
Show all eight activities
Attend a Python workshop focused on data engineering
Show steps
Advance your knowledge and skills in Python and data engineering by attending a specialized workshop led by industry experts.
Browse courses on
Python
Show steps
-
Research and identify relevant workshops
-
Register and attend the workshop
-
Actively participate in discussions, demos, and hands-on exercises
Solve Python coding challenges
Show steps
Challenge yourself with Python coding challenges to reinforce your grasp of data structures and algorithms.
Browse courses on
Python
Show steps
-
Join an online coding platform like LeetCode or HackerRank
-
Select coding challenges appropriate to your skill level
-
Solve the challenges, referring to documentation or online forums when needed
Create a Python script to automate a data processing task
Show steps
Solidify your Python skills by creating a project that leverages data manipulation and automation techniques.
Browse courses on
Python
Show steps
-
Identify a data processing task that can be automated
-
Design a solution using Python and appropriate libraries
-
Write the Python script, incorporating functions and data structures as needed
-
Test and refine the script to ensure accurate and efficient data processing
Develop a dashboard to visualize data using Python
Show steps
Combine your Python skills with data visualization techniques to create an interactive dashboard that effectively communicates data insights.
Browse courses on
Python
Show steps
-
Gather and clean the necessary data
-
Choose an appropriate Python library for data visualization
-
Design and develop the dashboard layout and visualizations
-
Deploy the dashboard for sharing and presentation
Participate in a Python-based hackathon or competition
Show steps
Engage in a competitive environment to showcase your Python skills and expand your knowledge in practical applications of data science or machine learning.
Browse courses on
Python
Show steps
-
Identify and register for an appropriate hackathon or competition
-
Form a team or work individually on the challenge
-
Develop a solution using Python and relevant technologies
-
Present or submit your solution for evaluation
Join a study group or online forum for Python and data engineering discussions
Show steps
Engage with peers and experts in the field by actively participating in discussions, sharing knowledge, and collaborating on projects.
Browse courses on
Python
Show steps
Show steps
Show steps
- Identify and join relevant study groups or online forums
- Participate in discussions, ask questions, and share your insights
- Collaborate on projects or group challenges
Review basic Python skills
Show steps
Let's brush up on the fundamentals of Python programming to ensure you have the necessary foundational skills for this course.
Browse courses on
Python
Show steps
Show steps
Show steps
- Review variables and data types
- Practice writing and executing simple Python scripts
- Create a few basic functions and classes
Complete an online Python tutorial
Show steps
Deepen your understanding of Python by following a guided tutorial that covers data structures and their applications.
Browse courses on
Python
Show steps
Show steps
Show steps
- Identify a reputable online tutorial or course
- Follow the tutorial step-by-step, practicing the concepts
- Complete any exercises or assignments provided in the tutorial
Five other activities
Expand to see all activities and additional details
Show all eight activities
Attend a Python workshop focused on data engineering
Show steps
Advance your knowledge and skills in Python and data engineering by attending a specialized workshop led by industry experts.
Browse courses on
Python
Show steps
Show steps
Show steps
- Research and identify relevant workshops
- Register and attend the workshop
- Actively participate in discussions, demos, and hands-on exercises
Solve Python coding challenges
Show steps
Challenge yourself with Python coding challenges to reinforce your grasp of data structures and algorithms.
Browse courses on
Python
Show steps
Show steps
Show steps
- Join an online coding platform like LeetCode or HackerRank
- Select coding challenges appropriate to your skill level
- Solve the challenges, referring to documentation or online forums when needed
Create a Python script to automate a data processing task
Show steps
Solidify your Python skills by creating a project that leverages data manipulation and automation techniques.
Browse courses on
Python
Show steps
Show steps
Show steps
- Identify a data processing task that can be automated
- Design a solution using Python and appropriate libraries
- Write the Python script, incorporating functions and data structures as needed
- Test and refine the script to ensure accurate and efficient data processing
Develop a dashboard to visualize data using Python
Show steps
Combine your Python skills with data visualization techniques to create an interactive dashboard that effectively communicates data insights.
Browse courses on
Python
Show steps
Show steps
Show steps
- Gather and clean the necessary data
- Choose an appropriate Python library for data visualization
- Design and develop the dashboard layout and visualizations
- Deploy the dashboard for sharing and presentation
Participate in a Python-based hackathon or competition
Show steps
Engage in a competitive environment to showcase your Python skills and expand your knowledge in practical applications of data science or machine learning.
Browse courses on
Python
Show steps
Show steps
Show steps
- Identify and register for an appropriate hackathon or competition
- Form a team or work individually on the challenge
- Develop a solution using Python and relevant technologies
- Present or submit your solution for evaluation
Career center
Learners who complete Scripting with Python and SQL for Data Engineering will develop knowledge and skills
that may be useful to these careers:
For more career information including salaries, visit:
OpenCourser.com/course/iw9sje/scripting
Reading list
We haven't picked any books for this reading list yet.
For more information about how these books relate to this course, visit:
OpenCourser.com/course/iw9sje/scripting
Share
Similar courses
Here are nine courses similar to
Scripting with Python and SQL for Data Engineering.
Data Engineering for Beginners with Python and SQL
OpenCourser.com/course/3l5eqo/data
Data Engineering for Beginners with Python and SQL
Most relevant
Scripting with Python and SQL for Data Engineering
OpenCourser.com/course/ll17jb/scripting
Scripting with Python and SQL for Data Engineering
Most relevant
Importing and Exporting Oracle Data for Developers
OpenCourser.com/course/4sf8wn/importing
Importing and Exporting Oracle Data for Developers
Most relevant
Essential SQL: Azure Data Factory and Data Engineering
OpenCourser.com/course/pgyj7t/essential
Essential SQL: Azure Data Factory and Data Engineering
Most relevant
Dimensional Modeling on the Microsoft SQL Server Platform
OpenCourser.com/course/8cewvf/dimensional
Dimensional Modeling on the Microsoft SQL Server Platform
Most relevant
Data Management with Databricks: Big Data with Delta Lakes
OpenCourser.com/course/4wz4sw/data
Data Management with Databricks: Big Data with Delta Lakes
Most relevant
Data Engineering Essentials using SQL, Python, and PySpark
OpenCourser.com/course/t0sfb9/data
Data Engineering Essentials using SQL, Python, and PySpark
Most relevant
Data Engineering Capstone Project
OpenCourser.com/course/c936bx/data
Data Engineering Capstone Project
Most relevant
Queries with OpenAI: Translate Natural Text to SQL
OpenCourser.com/course/rm9ske/queries
Queries with OpenAI: Translate Natural Text to SQL
Effort
3 to 6 hours per week for 4 weeks
Level
Introductory
Via
edX
Institution
Pragmatic AI Labs
Instructor
Alfredo Deza
Session
March 31, 2024
- March 30, 2025
Language
English
Transcript
English
Good to know
Designed for beginners, this course lays the foundational knowledge and core skills for those seeking to enter the field of data engineering
Taught by experienced professionals in data engineering, this course conveys real-world applications and practical techniques
Delves into the practical applications of Python, Bash, and SQL, equipping learners with industry-standard tools for data analysis and engineering
May require supplemental learning for those with limited programming experience, as it assumes some prior knowledge
Our mission
OpenCourser helps millions of learners each year. People visit us to learn workspace skills, ace their exams, and nurture their curiosity.
Our extensive catalog contains over 50,000 courses and twice as many books. Browse by search, by topic, or even by career interests. We'll match you to the right resources quickly.
Find this site helpful? Tell a friend about us.
Affiliate disclosure
We're supported by our community of learners. When you purchase or subscribe to courses and programs or purchase books, we may earn a commission from our partners.
Your purchases help us maintain our catalog and keep our servers humming without ads.
Thank you for supporting OpenCourser.
© 2016 - 2024 OpenCourser