Self-Driving Cars with Duckietown from edX

Robotics and AI are all around us and promise to revolutionize our daily lives. Autonomous vehicles have a huge potential to impact society in the near future, for example, by making owning private vehicles unnecessary!

Have you ever wondered how autonomous cars actually work?

With this course, you will start from a box of parts and finish with a scaled self-driving car that drives autonomously in your living room. In the process, you will use state-of-the-art approaches, the latest software tools, and real hardware in an engaging hands-on learning experience.

Self-driving cars with Duckietown is a practical introduction to vehicle autonomy. It explores real-world solutions to the theoretical challenges of autonomy, including their translation into algorithms and their deployment in simulation as well as on hardware.

Using modern software architectures built with Python, Robot Operating System (ROS), and Docker, you will appreciate the complementary strengths of classical architectures and modern machine learning-based approaches. The scope of this introductory course is to go from zero to having a self-driving car safely driving in a Duckietown.

This course is presented by Professors and Scientists who are passionate about robotics and accessible education. It uses the Duckietown robotic ecosystem, an open-source platform created at the MIT Computer Science and Artificial Intelligence Laboratory and now used by over 150 universities worldwide.

We support a track for learners to deploy their solutions in a simulation environment, and an additional option for learners that want to engage in the challenging but rewarding, tangible, hands-on learning experience of making the theory come to life in the real world. The hardware track is streamlined through an all-inclusive low-cost Jetson Nano-powered Duckiebot kit, inclusive of city track, available here.

This course is made possible thanks to the support of the Swiss Federal Institute of Technology in Zurich (ETH Zurich), in collaboration with the University of Montreal (Prof. Liam Paull), the Duckietown Foundation, and the Toyota Technological Institute at Chicago (Prof. Matthew Walter).

What's inside

Learning objectives

After this course, you will be able to program your duckiebots to navigate (without accidents!) in road lanes of a model city with rubber-duckie-pedestrian-obstacles using predominantly computer vision-based techniques.
Moreover, you will:
Recognize essential robot subsystems (sensing, actuation, computation, memory, mechanical) and describe their functions
Make your duckiebot drive in user-specified paths
Understand how to command a robot to reach a goal position
Make your duckiebot take driving decisions autonomously according to "traditional approaches", i.e., following the estimation, planning, control architecture
Make your duckiebot take driving decisions autonomously according to "modern approaches" (reinforcement learning)
Process streams of images
Be able to set up an efficient software environment for robotics with state-of-the-art tools (docker, ros, python)

Program your duckiebot and make it safely drive in empty roads lanes
Program your duckiebot and make it recognize and avoid rubber duckie obstacles
Submit your robot agents (a.k.a. "robot minds") to public challenges, and test your skills against your peers
Additional goals (require hardware)
Independently assemble a duckiebot and a duckietown
Remotely operate your duckiebot and see with its eye(s)
Be able to discuss differences between theory, simulation, and real word implementation for different approaches
Experience the challenges of deploying complex autonomous robots in the real world, and reap the rewards of getting it to work

After this course, you will be able to program your duckiebots to navigate (without accidents!) in road lanes of a model city with rubber-duckie-pedestrian-obstacles using predominantly computer vision-based techniques.
Moreover, you will:
Recognize essential robot subsystems (sensing, actuation, computation, memory, mechanical) and describe their functions
Make your duckiebot drive in user-specified paths
Understand how to command a robot to reach a goal position
Make your duckiebot take driving decisions autonomously according to "traditional approaches", i.e., following the estimation, planning, control architecture
Make your duckiebot take driving decisions autonomously according to "modern approaches" (reinforcement learning)
Process streams of images
Be able to set up an efficient software environment for robotics with state-of-the-art tools (docker, ros, python)
Program your duckiebot and make it safely drive in empty roads lanes
Program your duckiebot and make it recognize and avoid rubber duckie obstacles
Submit your robot agents (a.k.a. "robot minds") to public challenges, and test your skills against your peers
Additional goals (require hardware)
Independently assemble a duckiebot and a duckietown
Remotely operate your duckiebot and see with its eye(s)
Be able to discuss differences between theory, simulation, and real word implementation for different approaches
Experience the challenges of deploying complex autonomous robots in the real world, and reap the rewards of getting it to work

Syllabus

Module 0: Welcome to the course

Welcome to the course, by Prof. Emilio Frazzoli

You will familiarize yourself with the logistics and navigation interface of the course resources

You will start a learning journey in the world of robot autonomy with Duckietown

Module 1: Introduction to self-driving cars

The potentials and challenges

Levels of autonomy

The vision for autonomous vehicles (AVs)

Activities: You will set up your learning environment, and your Duckiebot, and make your first challenge submission

Module 2: Towards autonomy

Making a robot

Sensorimotor architectures

Stateful architectures

Logical and physical architectures

Application: You will create a reactive "Braitenberg" agent to avoid duckies and see how your agent compares to other submissions

Module 3: Modeling and Control

Introduction to control systems

Representations and models

PID control

Application: You will design an odometry function and PID controller to command your Duckiebot's angular velocity

Module 4: Robot Vision

Introduction to projective geometry

Camera modeling and calibration

Image processing

Application: You will develop image processing techniques necessary for visual lane servoing - controlling your Duckiebot to drive within markings

Module 5: Object Detection

Introduction to neural networks

Convolutional neural networks

One and two-stage object detection

Application: You will train a convolutional neural network (CNN) to detect duckies and integrate your model with ROS to run onboard your Duckiebot and avoid duckies

Module 6: State Estimation and Localization

Bayes filtering framework

Parameterized methods (Kalman filter)

Sampling-based methods (particle and histogram filter)

Application: You will build a state estimation algorithm combining the dynamics and sensor data of your Duckiebot in order to predict its pose as it travels through the world

Module 7: Planning I

Formalization of the planning problem

Application: You will create a collision checker to determine if your Duckiebot is crashing into an obstacle

Module 8: Planning II

Graphs

Graph search algorithms

Application: You will tackle a variety of path-planning challenges and leverage all the skills you've built thus far to navigate your Duckiebot in a variety of simulated environments

Module 9: Learning by Reinforcement

Markov decision processes

Value functions

Policy gradients

Domain randomization

Application: You will explore the capabilities and limitations of reinforcement learning models when applied to real-world robotics tasks such as lane following

Good to know

Know what's good

, what to watch for

, and possible dealbreakers

Teaches learners about self-driving cars, which is a highly active and transformative field

Instructors are recognized for their work in robotics and autonomous vehicle development

Uses Docker, ROS, and Python, which are industry-standard robotics tools and software

Offers hardware track that gives hands-on experience with physical robotics

Could enhance learning experience by providing more in-depth reinforcement learning content

Hardware track requires additional purchase of Duckiebot kit, which may pose a cost barrier for some learners

Activities

Be better prepared before your course. Deepen your understanding during and after it. Supplement your coursework and achieve mastery of the topics covered in Self-Driving Cars with Duckietown with these activities:

Review Linear Algebra

Show steps

Review the concepts of linear algebra to strengthen your mathematical foundation for the course.

Browse courses on Linear Algebra

Show steps

Go through your notes or textbook to refresh your memory on the basics of linear algebra.
Solve practice problems to test your understanding of the concepts.

Review Computer Vision Concepts

Show steps

Strengthen your understanding of computer vision concepts, which are crucial for self-driving cars to perceive and interpret their surroundings.

Browse courses on Computer Vision

Show steps

Review textbooks, online resources, or lecture notes on computer vision.
Focus on topics like image formation, feature extraction, and object detection.

Follow Tutorials on ROS and Gazebo

Show steps

Gain practical experience with the ROS software framework and Gazebo simulator, which are essential tools for developing and testing autonomous vehicles.

Browse courses on ROS

Show steps

Find online tutorials or courses on ROS and Gazebo.
Follow the tutorials step-by-step, building and running simulations of simple robots.
Experiment with different ROS topics and services to control and monitor your simulated robots.

Five other activities

Expand to see all activities and additional details

Show all eight activities

Solve Python Coding Challenges

Show steps

Improve your Python programming skills by solving coding challenges related to autonomous vehicle development.

Browse courses on Python

Show steps

Join online coding challenge platforms.
Select challenges that focus on topics relevant to autonomous vehicles, such as path planning, sensor data processing, and control algorithms.
Solve the challenges and compare your solutions with others.

Attend Industry Conferences on Autonomous Vehicles

Show steps

Connect with professionals in the autonomous vehicle industry, learn about the latest advancements, and explore potential career opportunities.

Browse courses on Networking

Show steps

Research upcoming conferences related to autonomous vehicles.
Register for the conference and prepare to actively participate in sessions and networking events.
Engage with speakers, attendees, and potential employers to expand your knowledge and professional network.

Develop a Simulation Scenario for Self-Driving Cars

Show steps

Design and implement a realistic simulation scenario that tests the capabilities of a self-driving car in various driving conditions.

Browse courses on Simulation

Show steps

Identify different driving scenarios that are challenging for self-driving cars.
Create a virtual environment using tools like CARLA or SUMO.
Develop realistic obstacles, traffic patterns, and weather conditions.
Test your scenario with different self-driving algorithms and evaluate their performance.

Participate in Open Source Robotics Projects

Show steps

Contribute to open source projects related to robotics and autonomous vehicles to gain practical experience and connect with the community.

Browse courses on Open Source

Show steps

Find open source projects on platforms like GitHub or GitLab.
Identify areas where you can contribute based on your skills and interests.
Join the project community and collaborate with other developers.

Mentor Students in Autonomous Vehicle Projects

Show steps

Share your knowledge and experience by mentoring students working on autonomous vehicle projects, fostering their growth and developing your own leadership skills.

Browse courses on Mentorship

Show steps

Join organizations or initiatives that connect mentors with students.
Identify students with projects related to autonomous vehicles.
Provide guidance, support, and feedback to help students achieve their project goals.

Career center

Learners who complete Self-Driving Cars with Duckietown will develop knowledge and skills that may be useful to these careers:

Autonomous Vehicle Systems Engineer

An Autonomous Vehicle Systems Engineer designs, develops, and tests the systems that control autonomous vehicles. These systems include sensors, actuators, and software. Autonomous Vehicle Systems Engineers work with a variety of engineers and technicians to create autonomous vehicles that are safe and reliable. This course may be useful for someone who wants to become an Autonomous Vehicle Systems Engineer because it provides a foundation in the basics of autonomous vehicles, including vehicle dynamics, control systems, and sensor fusion.

See salaries and explore the career path for Autonomous Vehicle Systems Engineer

Autonomous Vehicle Engineer

An Autonomous Vehicle Engineer designs, develops, and tests autonomous vehicles. These vehicles are capable of driving themselves without human input. Autonomous Vehicle Engineers work with a variety of technologies, such as sensors, actuators, and software, to create autonomous vehicles that are safe and reliable. This course may be useful for someone who wants to become an Autonomous Vehicle Engineer because it provides a foundation in the basics of autonomous vehicles, including vehicle dynamics, control systems, and sensor fusion.

See salaries and explore the career path for Autonomous Vehicle Engineer

Software Engineer

A Software Engineer designs, develops, and maintains software systems. They work with a variety of programming languages and technologies to create software that meets the needs of users. Software Engineers may work on a variety of projects, such as developing new features for existing software, creating new software applications, or maintaining and updating software systems. This course may be useful for someone who wants to become a Software Engineer because it provides a foundation in the basics of software engineering, including software design, programming, and testing.

See salaries and explore the career path for Software Engineer

Data Analyst

A Data Analyst collects, cleans, and analyzes data to identify trends and patterns. They use this information to help businesses make better decisions. Data Analysts may work with a variety of data sources, such as customer surveys, sales data, and social media data. This course may be useful for someone who wants to become a Data Analyst because it provides a foundation in the basics of data analysis, including data cleaning, data visualization, and statistics.

See salaries and explore the career path for Data Analyst

Robotics Engineer

A Robotics Engineer designs, builds, tests, and operates robots. They work with mechanical engineers to design the physical structure of the robot, electrical engineers to design the electrical systems, and computer scientists to design the software. Robotics Engineers may also work with other engineers to develop new technologies for robots, such as new sensors or actuators. This course may be useful for someone who wants to become a Robotics Engineer because it provides a foundation in the basics of robotics, including robot design, control, and programming.

See salaries and explore the career path for Robotics Engineer

Software Developer

A Software Developer designs, develops, and maintains software applications. They work with a variety of programming languages and technologies to create software that meets the needs of users. Software Developers may work on a variety of projects, such as developing new features for existing software, creating new software applications, or maintaining and updating software systems. This course may be useful for someone who wants to become a Software Developer because it provides a foundation in the basics of software engineering, including software design, programming, and testing.

See salaries and explore the career path for Software Developer

Computer Vision Engineer

A Computer Vision Engineer designs and develops computer vision systems. These systems use cameras and other sensors to capture and analyze images and videos. Computer Vision Engineers work on a variety of applications, such as facial recognition, object detection, and medical imaging. This course may be useful for someone who wants to become a Computer Vision Engineer because it provides a foundation in the basics of computer vision, including image processing, object detection, and machine learning.

See salaries and explore the career path for Computer Vision Engineer

Robotics Software Engineer

A Robotics Software Engineer designs, develops, and maintains software for robots. This software controls the robot's movement, sensors, and other systems. Robotics Software Engineers work with a variety of engineers and technicians to create robots that are safe and reliable. This course may be useful for someone who wants to become a Robotics Software Engineer because it provides a foundation in the basics of robotics, including robot design, control, and programming.

See salaries and explore the career path for Robotics Software Engineer

Robotics Technician

A Robotics Technician installs, maintains, and repairs robots. They work with a variety of robots, including industrial robots, medical robots, and military robots. Robotics Technicians may also work with other engineers and technicians to develop new robots or improve existing robots. This course may be useful for someone who wants to become a Robotics Technician because it provides a foundation in the basics of robotics, including robot design, control, and programming.

See salaries and explore the career path for Robotics Technician

Software Quality Assurance Analyst

A Software Quality Assurance Analyst tests software to ensure that it meets the requirements of the user. They work with developers to identify and fix bugs. Software Quality Assurance Analysts may also work with other stakeholders, such as product managers and customers, to ensure that the software meets the needs of users. This course may be useful for someone who wants to become a Software Quality Assurance Analyst because it provides a foundation in the basics of software testing, including test design, test execution, and test reporting.

See salaries and explore the career path for Software Quality Assurance Analyst

Machine Learning Scientist

A Machine Learning Scientist develops new machine learning algorithms and techniques. They work on a variety of problems, such as natural language processing, computer vision, and speech recognition. Machine Learning Scientists may also work with other researchers to develop new applications for machine learning, such as autonomous vehicles or medical diagnosis. This course may be useful for someone who wants to become a Machine Learning Scientist because it provides a foundation in the basics of machine learning, including model design, training, and evaluation.

See salaries and explore the career path for Machine Learning Scientist

Computer Vision Researcher

A Computer Vision Researcher develops new computer vision algorithms and techniques. They work on a variety of problems, such as object detection, image segmentation, and facial recognition. Computer Vision Researchers may also work with other researchers to develop new applications for computer vision, such as autonomous vehicles or medical imaging. This course may be useful for someone who wants to become a Computer Vision Researcher because it provides a foundation in the basics of computer vision, including image processing, object detection, and machine learning.

See salaries and explore the career path for Computer Vision Researcher

Data Scientist

A Data Scientist uses data to solve business problems. They collect, clean, and analyze data to identify trends and patterns. Data Scientists then use this information to develop models and algorithms that can be used to make predictions or recommendations. This course may be useful for someone who wants to become a Data Scientist because it provides a foundation in the basics of data science, including data analysis, machine learning, and statistics.

See salaries and explore the career path for Data Scientist

Computer Vision Applications Developer

A Computer Vision Applications Developer develops and deploys computer vision applications. These applications use computer vision algorithms to solve a variety of problems, such as object detection, image segmentation, and facial recognition. Computer Vision Applications Developers work with a variety of stakeholders, such as end users, product managers, and business analysts, to create computer vision applications that meet the needs of users. This course may be useful for someone who wants to become a Computer Vision Applications Developer because it provides a foundation in the basics of computer vision, including image processing, object detection, and machine learning.

See salaries and explore the career path for Computer Vision Applications Developer

Machine Learning Engineer

A Machine Learning Engineer designs and develops machine learning models. These models can be used to solve a variety of problems, such as predicting customer behavior, detecting fraud, or diagnosing diseases. Machine Learning Engineers work with a variety of data sources and technologies to create and deploy machine learning models. This course may be useful for someone who wants to become a Machine Learning Engineer because it provides a foundation in the basics of machine learning, including model design, training, and evaluation.

See salaries and explore the career path for Machine Learning Engineer