We may earn an affiliate commission when you visit our partners.
Course image
Krishna Sridhar

As AI moves beyond the cloud, on-device inference is rapidly expanding to smartphones, IoT devices, robots, AR/VR headsets, and more. Billions of mobile and other edge devices are ready to run optimized AI models.

Read more

As AI moves beyond the cloud, on-device inference is rapidly expanding to smartphones, IoT devices, robots, AR/VR headsets, and more. Billions of mobile and other edge devices are ready to run optimized AI models.

This course equips you with key skills to deploy AI on device:

1. Explore how deploying models on device reduces latency, enhances efficiency, and preserves privacy.

2. Go through key concepts of on-device deployment such as neural network graph capture, on-device compilation, and hardware acceleration.

3. Convert pretrained models from PyTorch and TensorFlow for on-device compatibility.

4. Deploy a real-time image segmentation model on device with just a few lines of code.

5. Test your model performance and validate numerical accuracy when deploying to on-device environments

6. Quantize and make your model up to 4x faster and 4x smaller for higher on-device performance.

7. See a demonstration of the steps for integrating the model into a functioning Android app.

Learn from Krishna Sridhar, Senior Director of Engineering at Qualcomm, who has played a pivotal role in deploying over 1,000 models on devices and, with his team, has created the infrastructure used by over 100,000 applications.

By learning these techniques, you’ll be positioned to develop and deploy AI to billions of devices and optimize your complex models to run efficiently on the edge.

Enroll now

What's inside

Syllabus

Untitled Module
As AI moves beyond the cloud, on-device inference is rapidly expanding to smartphones, IoT devices, robots, AR/VR headsets, and more. Billions of mobile and other edge devices are ready to run optimized AI models. This course equips you with key skills to deploy AI on device: 1. Explore how deploying models on device reduces latency, enhances efficiency, and preserves privacy. 2. Go through key concepts of on-device deployment such as neural network graph capture, on-device compilation, and hardware acceleration. 3. Convert pretrained models from PyTorch and TensorFlow for on-device compatibility. 4. Deploy a real-time image segmentation model on device with just a few lines of code. 5. Test your model performance and validate numerical accuracy when deploying to on-device environments 6. Quantize and make your model up to 4x faster and 4x smaller for higher on-device performance. 7. See a demonstration of the steps for integrating the model into a functioning Android app. Learn from Krishna Sridhar, Senior Director of Engineering at Qualcomm, who has played a pivotal role in deploying over 1,000 models on devices and, with his team, has created the infrastructure used by over 100,000 applications. By learning these techniques, you’ll be positioned to develop and deploy AI to billions of devices and optimize your complex models to run efficiently on the edge.

Good to know

Know what's good
, what to watch for
, and possible dealbreakers
Geared toward equipping learners with essential techniques to deploy machine learning models on devices
Taught by Krishna Sridhar, a recognized expert in on-device AI deployment with substantial experience in the field
Provides hands-on labs to reinforce learning and practical application of concepts
Aims to support learners in developing and deploying AI models efficiently on billions of devices
Covers key concepts in on-device AI deployment, including neural network graph capture and hardware acceleration
Suitable for learners interested in developing and deploying AI models specifically on devices

Save this course

Save Introduction to On-Device AI to your list so you can find it easily later:
Save

Activities

Be better prepared before your course. Deepen your understanding during and after it. Supplement your coursework and achieve mastery of the topics covered in Introduction to On-Device AI with these activities:
Review TensorFlow Library
Go through TensorFlow's documentation to refresh your understanding of the library, enabling you to effectively convert models.
Browse courses on Deep Learning
Show steps
  • Visit TensorFlow website and review their documentation
  • Practice writing TensorFlow code
Connect with On-Device AI Deployment Experts
Guidance from experienced professionals can accelerate your learning and provide valuable insights into the field of on-device AI deployment.
Show steps
  • Identify potential mentors who have expertise in on-device AI deployment
  • Reach out to these individuals and express your interest in mentorship
Practice Neural Network Graph Capture Techniques
Getting your hands dirty with these techniques will make you well-prepared for the core concepts of this course on on-device AI deployment.
Show steps
  • Read course materials about Neural Network Graph Capture
  • Implement a Neural Network Graph Capture function
  • Solve coding problems that require Neural Network Graph Capture
Ten other activities
Expand to see all activities and additional details
Show all 13 activities
Review models before course
Review key machine learning theory and concepts (i.e. model training, validation, evaluation) before the start of the course.
Browse courses on Machine Learning
Show steps
  • Review key theoretical concepts: Model Training
  • Review key theoretical concepts: Model Validation
  • Review key theoretical concepts: Model Evaluation
Practice Neural Network Optimization
Refresh your skills in neural network optimization to prepare for this course's focus on optimizing AI models for on-device deployment.
Show steps
  • Review your notes and assignments from previous machine learning or neural network optimization courses.
  • Follow tutorials on neural network optimization techniques.
  • Implement neural network optimization algorithms in a programming language of your choice.
Python C++ coding practice
Complete practice problems using Python and C++ to strengthen coding skills necessary for deploying models on device.
Browse courses on Python
Show steps
  • Identify online coding practice problems
  • Attempt to solve coding problems
  • Review solutions and identify areas for improvement
Follow a Tutorial on Model Performance Testing and Validation
Understanding how to test and validate models will increase your confidence while deploying AI on-device.
Show steps
  • Identify a guided tutorial that covers testing and validating models for on-device environments
  • Follow the tutorial and complete all exercises
  • Apply the techniques you learned to your own project
Explore On-Device AI Tutorials
Supplement your learning by exploring tutorials and resources on on-device AI to gain a deeper understanding of the concepts and techniques covered in this course.
Show steps
  • Search for tutorials on topics such as on-device model deployment, optimization, and hardware acceleration.
  • Follow tutorials that align with your interests and skill level.
  • Experiment with the code and examples provided in the tutorials.
On-device inference with MobileNet
Practice implementing on-device inference with MobileNet through online tutorials.
Browse courses on AI
Show steps
  • Identify tutorial on on-device inference with MobileNet
  • Follow the steps outlined in the tutorial
  • Test and evaluate the accuracy of your implementation
AI Model Deployment in Android
Build an Android application that integrates an AI model for image recognition, putting your theoretical knowledge into practice.
Browse courses on Android Development
Show steps
  • Plan and design the Android application
  • Develop the Android application
  • Integrate the AI model into the application
  • Test and evaluate the application
Deploy a Real-Time Object Detection Model on a Smartphone
Hands-on experience is one of the best ways to solidify the concepts learned in this course on on-device AI deployment.
Show steps
  • Gather the necessary hardware and software
  • Follow the course materials to deploy a real-time object detection model
  • Test and evaluate the deployed model
Technical Blog on Model Quantization
Write a blog post explaining the process and effectiveness of quantizing models, to reinforce knowledge and showcase understanding.
Browse courses on Software Development
Show steps
  • Research model quantization
  • Write a technical blog describing your findings
  • Publish your blog and share it with others
Attend a Workshop on Quantization for On-Device AI Deployment
Workshops offer an immersive learning experience that can greatly enhance your understanding of on-device AI deployment.
Show steps
  • Identify and register for a workshop on quantization for on-device AI deployment
  • Attend the workshop and participate actively
  • Apply the knowledge gained from the workshop to your own projects

Career center

Learners who complete Introduction to On-Device AI will develop knowledge and skills that may be useful to these careers:

Reading list

We haven't picked any books for this reading list yet.

Share

Help others find this course page by sharing it with your friends and followers:

Similar courses

Here are nine courses similar to Introduction to On-Device AI.
Landing.AI for Beginners: Build Data Visualization AI...
Most relevant
AWS IoT: Developing and Deploying an Internet of Things
Most relevant
LLMOps & ML Deployment: Bring LLMs and GenAI to Production
Most relevant
Quantization Fundamentals with Hugging Face
Most relevant
Large Language Models with Azure
Most relevant
Deploying TinyML
Most relevant
Business Considerations for Edge Computing
Most relevant
Generative AI and LLMs on AWS
Most relevant
IoT Devices
Most relevant
Our mission

OpenCourser helps millions of learners each year. People visit us to learn workspace skills, ace their exams, and nurture their curiosity.

Our extensive catalog contains over 50,000 courses and twice as many books. Browse by search, by topic, or even by career interests. We'll match you to the right resources quickly.

Find this site helpful? Tell a friend about us.

Affiliate disclosure

We're supported by our community of learners. When you purchase or subscribe to courses and programs or purchase books, we may earn a commission from our partners.

Your purchases help us maintain our catalog and keep our servers humming without ads.

Thank you for supporting OpenCourser.

© 2016 - 2024 OpenCourser