Introduction to High-Performance and Parallel Computing from Coursera

This course introduces the fundamentals of high-performance and parallel computing. It is targeted to scientists, engineers, scholars, really everyone seeking to develop the software skills necessary for work in parallel software environments. These skills include big-data analysis, machine learning, parallel programming, and optimization. We will cover the basics of Linux environments and bash scripting all the way to high throughput computing and parallelizing code. We recommend you are familiar with either Fortran 90, C++, or Python to complete some of the programming assignments.

After completing this course, you will familiar with:

*The components of a high-performance distributed computing system

*Types of parallel programming models and the situations in which they might be used

*High-throughput computing

*Shared memory parallelism

*Distributed memory parallelism

*Navigating a typical Linux-based HPC environment

*Assessing and analyzing application scalability including weak and strong scaling

*Quantifying the processing, data, and cost requirements for a computational project or workflow

This course can be taken for academic credit as part of CU Boulder’s Master of Science in Data Science (MS-DS) degree offered on the Coursera platform. The MS-DS is an interdisciplinary degree that brings together faculty from CU Boulder’s departments of Applied Mathematics, Computer Science, Information Science, and others. With performance-based admissions and no application process, the MS-DS is ideal for individuals with a broad range of undergraduate education and/or professional experience in computer science, information science, mathematics, and statistics. Learn more about the MS-DS program at https://www.coursera.org/degrees/master-of-science-data-science-boulder.

What's inside

Syllabus

High-Performance Computing (HPC) for Non-Computer Scientists

Get to know the basics of an HPC system. Users will learn how to work with common high performance computing systems they may encounter in future efforts. This includes navigating filesystems, working with a typical HPC operating system (Linux), and some of the basic concepts of HPC. We will also provide users some key information that is specific to the logistics of this course.

Traffic lights

Read about what's good

what should give you pause

and possible dealbreakers

Develops skills for big-data analysis, machine learning, parallel programming, and optimization, which are core skills for data scientists

Taught by Shelley Knuth and Thomas Hauser, who are recognized for their work in high-performance computing

Examines high-throughput computing, parallel programming, and other topics highly relevant to data science and computer science

Students are advised to have familiarity with either Fortran 90, C++, or Python to complete some of the programming assignments

Reviews summary

Foundational hpc and parallel computing skills

According to students, this course provides a solid and practical introduction to high-performance and parallel computing, making it highly relevant for career-focused professionals in fields like data science, machine learning, and engineering. Learners appreciate its focus on developing essential software skills for parallel environments, including navigating Linux HPC systems and optimizing code. A prior familiarity with programming languages such as Fortran 90, C++, or Python is recommended to fully benefit from the hands-on assignments. The course is seen as a well-structured stepping stone for practical application.

Highly relevant for professionals in data science and engineering.

"As an engineer, I found this course directly applicable to my professional development."

"It's a great course for anyone in data science looking to improve their parallel computing skills."

"The content aligns well with the demands of big-data analysis and machine learning roles."

Focuses on applicable skills for real-world HPC environments.

"The material on navigating Linux HPC environments was immediately useful for my work."

"I learned practical skills for optimizing and parallelizing code that I can apply."

"This course equipped me with the software skills needed for big-data and machine learning applications."

Provides essential knowledge for HPC and parallel programming.

"I gained a solid foundation in the basics of high-performance computing."

"This course is excellent for anyone looking to understand core HPC concepts."

"It really helped me grasp the fundamentals of parallel programming and memory nuances."

Familiarity with C++, Fortran, or Python is beneficial.

"I found that having a good grasp of Python prior to the course was really helpful for the assignments."

"Without some programming experience, I imagine parts of the course would be quite challenging."

"It's important to have basic coding skills in C++ or Python to fully benefit from the programming exercises."

Activities

Be better prepared before your course. Deepen your understanding during and after it. Supplement your coursework and achieve mastery of the topics covered in Introduction to High-Performance and Parallel Computing with these activities:

Organize and review notes, assignments, and quizzes from previous courses related to high-performance computing

Show steps

Helps refresh knowledge from previous courses and reinforces concepts.

Browse courses on High-Performance Computing

Show steps

Gather notes, assignments, and quizzes from previous courses.
Review and summarize the key concepts covered.

Review basic concepts in Linux operating systems and bash scripting

Show steps

Strengthens foundational skills in operating systems and scripting, which are essential for working in high-performance computing environments.

Browse courses on Linux

Show steps

Go through online tutorials or documentation on Linux and bash scripting.
Practice using Linux commands and writing simple bash scripts.

Review Thomas Sterling's 'High Performance Computing: Modern Systems and Practices'

Show steps

Provides a concise introduction to high-performance and parallel computing.

View High Performance Computing: Modern Systems and... on Amazon

Show steps

Six other activities

Expand to see all activities and additional details

Show all nine activities

Participate in study groups or online forums dedicated to high-performance computing

Show steps

Provides opportunities to interact with peers, share knowledge, and get support.

Show steps

Identify and join a study group or online forum.
Participate in discussions, ask questions, and offer help to others.

Complete Introduction to Parallel Programming in C++ on the NVIDIA CUDA platform

Show steps

Introduces parallel programming concepts using C++ and the NVIDIA CUDA platform, which are commonly used in high-performance computing.

Browse courses on CUDA

Show steps

Follow the online tutorials and complete the exercises.
Experiment with different code optimizations and compare performance results.

Attend workshops on topics related to high-performance computing

Show steps

Provides an opportunity to learn from experts and network with professionals in the field.

Show steps

Identify and register for relevant workshops.
Attend the workshops and actively participate in discussions.

Solve parallel programming exercises on platforms like LeetCode or HackerRank

Show steps

Provides hands-on practice in solving parallel programming problems.

Browse courses on Parallel Programming

Show steps

Choose a platform like LeetCode or HackerRank.
Select and solve parallel programming exercises.
Analyze and optimize your solutions for efficiency.

Contribute to open-source projects related to high-performance computing

Show steps

Provides exposure to real-world high-performance computing projects and contributes to the community.

Browse courses on Open Source

Show steps

Identify open-source projects related to high-performance computing.
Choose a project and understand its goals and codebase.
Contribute to the project by fixing bugs, adding features, or improving documentation.

Develop a parallel program to solve a real-world problem

Show steps

Applies parallel programming concepts to solve practical problems and experience the benefits firsthand.

Browse courses on Parallel Programming

Show steps

Identify a suitable problem that can benefit from parallelization.
Design and implement a parallel algorithm.
Test and evaluate the performance of your program.

Career center

Learners who complete Introduction to High-Performance and Parallel Computing will develop knowledge and skills that may be useful to these careers:

High-Performance Computing Architect

High-Performance Computing Architects are responsible for designing, implementing, and maintaining high-performance computing (HPC) systems. They work with scientists, engineers, and other users to understand their computational needs and develop solutions that meet those needs. They also work with hardware and software vendors to ensure that the HPC systems they design are compatible with the latest technologies. This course provides a solid foundation in the principles of HPC, including parallel programming, performance optimization, and system architecture. It also provides hands-on experience with HPC systems, which will be invaluable for anyone who wants to pursue a career in HPC.

See salaries and explore the career path for High-Performance Computing Architect

Parallel Programming Engineer

Parallel Programming Engineers are responsible for developing and maintaining parallel programs. They work with scientists, engineers, and other users to understand their computational needs and develop parallel programs that meet those needs. They also work with hardware and software vendors to ensure that the parallel programs they develop are compatible with the latest technologies. This course provides a solid foundation in the principles of parallel programming, including parallel algorithms, data structures, and performance optimization. It also provides hands-on experience with parallel programming languages and tools, which will be invaluable for anyone who wants to pursue a career in parallel programming.

See salaries and explore the career path for Parallel Programming Engineer

High Performance Computing Specialist

High-Performance Computing Specialists are responsible for managing and supporting HPC systems. They work with users to provide training and support, and they work with hardware and software vendors to ensure that the HPC systems are running smoothly. They also work with scientists, engineers, and other users to help them develop and optimize their computational applications. This course provides a solid foundation in the principles of HPC, including system administration, performance optimization, and application development. It also provides hands-on experience with HPC systems, which will be invaluable for anyone who wants to pursue a career in HPC.

See salaries and explore the career path for High Performance Computing Specialist

Computational Scientist

Computational Scientists use computers to solve scientific problems. They develop and use mathematical models and simulations to study a wide range of scientific phenomena, including weather patterns, climate change, and the behavior of molecules. They also work with experimental scientists to design and conduct experiments, and they analyze data to draw conclusions about the natural world. This course provides a solid foundation in the principles of computational science, including numerical methods, data analysis, and visualization. It also provides hands-on experience with computational science tools and techniques, which will be invaluable for anyone who wants to pursue a career in computational science.

See salaries and explore the career path for Computational Scientist

Data Scientist

Data Scientists use data to solve business problems. They collect, clean, and analyze data to identify trends and patterns. They also develop and use machine learning models to make predictions and recommendations. This course provides a solid foundation in the principles of data science, including data analysis, machine learning, and visualization. It also provides hands-on experience with data science tools and techniques, which will be invaluable for anyone who wants to pursue a career in data science.

See salaries and explore the career path for Data Scientist

Machine Learning Engineer

Machine Learning Engineers develop and maintain machine learning models. They work with data scientists and other users to understand their machine learning needs and develop models that meet those needs. They also work with hardware and software vendors to ensure that the machine learning models they develop are compatible with the latest technologies. This course provides a solid foundation in the principles of machine learning, including machine learning algorithms, data structures, and performance optimization. It also provides hands-on experience with machine learning languages and tools, which will be invaluable for anyone who wants to pursue a career in machine learning.

See salaries and explore the career path for Machine Learning Engineer

Software Engineer

This course may be useful for Software Engineers who want to develop software for high-performance computing systems. The course provides a solid foundation in the principles of HPC, including parallel programming, performance optimization, and system architecture. It also provides hands-on experience with HPC systems, which will be invaluable for anyone who wants to develop software for HPC.

See salaries and explore the career path for Software Engineer

Computer Scientist

This course may be useful for Computer Scientists who want to specialize in high-performance computing. The course provides a solid foundation in the principles of HPC, including parallel programming, performance optimization, and system architecture. It also provides hands-on experience with HPC systems, which will be invaluable for anyone who wants to pursue a career in HPC.

See salaries and explore the career path for Computer Scientist

Systems Analyst

This course may be useful for Systems Analysts who want to work with high-performance computing systems. The course provides a solid foundation in the principles of HPC, including system administration, performance optimization, and application development. It also provides hands-on experience with HPC systems, which will be invaluable for anyone who wants to work with HPC systems.

See salaries and explore the career path for Systems Analyst

Database Administrator

This course may be useful for Database Administrators who want to work with high-performance computing systems. The course provides a solid foundation in the principles of HPC, including system administration, performance optimization, and application development. It also provides hands-on experience with HPC systems, which will be invaluable for anyone who wants to work with HPC systems.

See salaries and explore the career path for Database Administrator

Network Administrator

This course may be useful for Network Administrators who want to work with high-performance computing systems. The course provides a solid foundation in the principles of HPC, including system administration, performance optimization, and application development. It also provides hands-on experience with HPC systems, which will be invaluable for anyone who wants to work with HPC systems.

See salaries and explore the career path for Network Administrator

Security Analyst

This course may be useful for Security Analysts who want to work with high-performance computing systems. The course provides a solid foundation in the principles of HPC, including system administration, performance optimization, and application development. It also provides hands-on experience with HPC systems, which will be invaluable for anyone who wants to work with HPC systems.

See salaries and explore the career path for Security Analyst

Business Analyst

This course may be useful for Business Analysts who want to understand the potential of high-performance computing for their businesses. The course provides a solid foundation in the principles of HPC, including its benefits, challenges, and applications. It also provides hands-on experience with HPC systems, which will be invaluable for anyone who wants to make informed decisions about using HPC for their businesses.

See salaries and explore the career path for Business Analyst

Project Manager

This course may be useful for Project Managers who are working on projects that involve high-performance computing. The course provides a solid foundation in the principles of HPC, including its benefits, challenges, and applications. It also provides hands-on experience with HPC systems, which will be invaluable for anyone who wants to successfully manage projects that involve HPC.

See salaries and explore the career path for Project Manager

Technical Writer

This course may be useful for Technical Writers who want to write about high-performance computing. The course provides a solid foundation in the principles of HPC, including its benefits, challenges, and applications. It also provides hands-on experience with HPC systems, which will be invaluable for anyone who wants to write clear and accurate documentation about HPC.

See salaries and explore the career path for Technical Writer