Introduction to High-Performance and Parallel Computing from Coursera

This course introduces the fundamentals of high-performance and parallel computing. It is targeted to scientists, engineers, scholars, really everyone seeking to develop the software skills necessary for work in parallel software environments. These skills include big-data analysis, machine learning, parallel programming, and optimization. We will cover the basics of Linux environments and bash scripting all the way to high throughput computing and parallelizing code. We recommend you are familiar with either Fortran 90, C++, or Python to complete some of the programming assignments.

After completing this course, you will familiar with:

*The components of a high-performance distributed computing system

*Types of parallel programming models and the situations in which they might be used

*High-throughput computing

*Shared memory parallelism

*Distributed memory parallelism

*Navigating a typical Linux-based HPC environment

*Assessing and analyzing application scalability including weak and strong scaling

*Quantifying the processing, data, and cost requirements for a computational project or workflow

This course can be taken for academic credit as part of CU Boulder’s Master of Science in Data Science (MS-DS) degree offered on the Coursera platform. The MS-DS is an interdisciplinary degree that brings together faculty from CU Boulder’s departments of Applied Mathematics, Computer Science, Information Science, and others. With performance-based admissions and no application process, the MS-DS is ideal for individuals with a broad range of undergraduate education and/or professional experience in computer science, information science, mathematics, and statistics. Learn more about the MS-DS program at https://www.coursera.org/degrees/master-of-science-data-science-boulder.

What's inside

Syllabus

High-Performance Computing (HPC) for Non-Computer Scientists

Get to know the basics of an HPC system. Users will learn how to work with common high performance computing systems they may encounter in future efforts. This includes navigating filesystems, working with a typical HPC operating system (Linux), and some of the basic concepts of HPC. We will also provide users some key information that is specific to the logistics of this course.

Nuts and Bolts of HPC

During this week we will actually begin to use HPC infrastructure. Some concepts we will learn are - how to load software appropriately onto an HPC system, what the different types of nodes a user can expect to encounter on a system, and how to submit a job to conduct work, such as perform calculations.

Basic Parallelism

In this module, we will introduce users to the nuances of memory on a high performance computing system. We will also cover some ways to conduct work on a system most efficiently. We will also introduce some beginning components of parallel programming.

Evaluating Parallel Program Performance

In this module, we will continue to review topics related to using a high performance computing system most efficiently, including scaling your workflow measuring how efficient your work on a system is, and how to utilize as much of the computing resource as possible.

Good to know

Know what's good

, what to watch for

, and possible dealbreakers

Develops skills for big-data analysis, machine learning, parallel programming, and optimization, which are core skills for data scientists

Taught by Shelley Knuth and Thomas Hauser, who are recognized for their work in high-performance computing

Examines high-throughput computing, parallel programming, and other topics highly relevant to data science and computer science

Students are advised to have familiarity with either Fortran 90, C++, or Python to complete some of the programming assignments

Reviews summary

Hpc concepts, assignments emphasized

Learners say that this Introduction to High-Performance and Parallel Computing course is a good one, despite some poorly written assignments. According to students, the assignments are either very easy or take too much time to figure out because the instructions are unclear and there is no support from course staff. As such, learners say that the majority of their time is spent trying to get the assignments in the correct format. However, this course does a good job of introducing basic HPC concepts and has interesting lectures.

Instructor provides interesting lectures

"I really liked the course; it's not too difficult or too easy, but you still have to think carefully when solving the assignments."

"I liked the course more and rated it higher."

"It's accessible for non-computer scientists without a relevant background to learn the basic concepts of HPC and parallel computing."

"The video content is good and I enjoyed the teacher, but this course should really be removed from Coursera until the technical issues are fixed."

Course provides a good overview of HPC

"The course gives a good intro into the HPC topic."

"Good for an introduction of HPC concepts, but some lectures leave a lot to be desired."

"Great basic overview, I finally understand what IT is talking about in meetings."

"As an introduction, it shows basic concepts of HPC."

Auto-grader may give unclear feedback

"It makes you waste too many hours trying to figure out what is wrong with your solution which turns out to be like a blank line at the end or you are even submitting something else."

"The quizzes are very easy (too easy?) and each quiz consists of ~5 questions."

"The programming assignments are also very, very basic."

"The automatic grader is an annoying problem especially for the timing module."

Assignment instructions are unclear

"The instructions in the week 4 assignments aren't really clear about what is expected in order for a learner to pass."

"I spent hours on incredibly stupid issues, like the Week 4 assignment needs a blank row on line four for the auto-grader to accept it."

"The problem instruction did not specify what they want you to do."

"Even questions that have been asked 3 months ago remained unanswered by the instructors who seem to be absent from the course."

Assignments either very easy or frustratingly unclear

"The assignments are either very, very basic."

"The actual task is to figure out what should be done and what the correct input format is."

"Some assignments cannot be passed if you don't imagine what the parser would expect, especially in terms of paths."

Activities

Be better prepared before your course. Deepen your understanding during and after it. Supplement your coursework and achieve mastery of the topics covered in Introduction to High-Performance and Parallel Computing with these activities:

Organize and review notes, assignments, and quizzes from previous courses related to high-performance computing

Show steps

Helps refresh knowledge from previous courses and reinforces concepts.

Browse courses on High-Performance Computing

Show steps

Gather notes, assignments, and quizzes from previous courses.
Review and summarize the key concepts covered.

Review basic concepts in Linux operating systems and bash scripting

Show steps

Strengthens foundational skills in operating systems and scripting, which are essential for working in high-performance computing environments.

Browse courses on Linux

Show steps

Go through online tutorials or documentation on Linux and bash scripting.
Practice using Linux commands and writing simple bash scripts.

Review Thomas Sterling's 'High Performance Computing: Modern Systems and Practices'

Show steps

Provides a concise introduction to high-performance and parallel computing.

View High Performance Computing: Modern Systems and... on Amazon

Show steps

Six other activities

Expand to see all activities and additional details

Show all nine activities

Participate in study groups or online forums dedicated to high-performance computing

Show steps

Provides opportunities to interact with peers, share knowledge, and get support.

Show steps

Identify and join a study group or online forum.
Participate in discussions, ask questions, and offer help to others.

Complete Introduction to Parallel Programming in C++ on the NVIDIA CUDA platform

Show steps

Introduces parallel programming concepts using C++ and the NVIDIA CUDA platform, which are commonly used in high-performance computing.

Browse courses on CUDA

Show steps

Follow the online tutorials and complete the exercises.
Experiment with different code optimizations and compare performance results.

Attend workshops on topics related to high-performance computing

Show steps

Provides an opportunity to learn from experts and network with professionals in the field.

Show steps

Identify and register for relevant workshops.
Attend the workshops and actively participate in discussions.

Solve parallel programming exercises on platforms like LeetCode or HackerRank

Show steps

Provides hands-on practice in solving parallel programming problems.

Browse courses on Parallel Programming

Show steps

Choose a platform like LeetCode or HackerRank.
Select and solve parallel programming exercises.
Analyze and optimize your solutions for efficiency.

Contribute to open-source projects related to high-performance computing

Show steps

Provides exposure to real-world high-performance computing projects and contributes to the community.

Browse courses on Open Source

Show steps

Identify open-source projects related to high-performance computing.
Choose a project and understand its goals and codebase.
Contribute to the project by fixing bugs, adding features, or improving documentation.

Develop a parallel program to solve a real-world problem

Show steps

Applies parallel programming concepts to solve practical problems and experience the benefits firsthand.

Browse courses on Parallel Programming

Show steps

Identify a suitable problem that can benefit from parallelization.
Design and implement a parallel algorithm.
Test and evaluate the performance of your program.

Career center

Learners who complete Introduction to High-Performance and Parallel Computing will develop knowledge and skills that may be useful to these careers:

High-Performance Computing Architect

High-Performance Computing Architects are responsible for designing, implementing, and maintaining high-performance computing (HPC) systems. They work with scientists, engineers, and other users to understand their computational needs and develop solutions that meet those needs. They also work with hardware and software vendors to ensure that the HPC systems they design are compatible with the latest technologies. This course provides a solid foundation in the principles of HPC, including parallel programming, performance optimization, and system architecture. It also provides hands-on experience with HPC systems, which will be invaluable for anyone who wants to pursue a career in HPC.

See salaries and explore the career path for High-Performance Computing Architect

Parallel Programming Engineer

Parallel Programming Engineers are responsible for developing and maintaining parallel programs. They work with scientists, engineers, and other users to understand their computational needs and develop parallel programs that meet those needs. They also work with hardware and software vendors to ensure that the parallel programs they develop are compatible with the latest technologies. This course provides a solid foundation in the principles of parallel programming, including parallel algorithms, data structures, and performance optimization. It also provides hands-on experience with parallel programming languages and tools, which will be invaluable for anyone who wants to pursue a career in parallel programming.

See salaries and explore the career path for Parallel Programming Engineer

High Performance Computing Specialist

High-Performance Computing Specialists are responsible for managing and supporting HPC systems. They work with users to provide training and support, and they work with hardware and software vendors to ensure that the HPC systems are running smoothly. They also work with scientists, engineers, and other users to help them develop and optimize their computational applications. This course provides a solid foundation in the principles of HPC, including system administration, performance optimization, and application development. It also provides hands-on experience with HPC systems, which will be invaluable for anyone who wants to pursue a career in HPC.

See salaries and explore the career path for High Performance Computing Specialist

Computational Scientist

Computational Scientists use computers to solve scientific problems. They develop and use mathematical models and simulations to study a wide range of scientific phenomena, including weather patterns, climate change, and the behavior of molecules. They also work with experimental scientists to design and conduct experiments, and they analyze data to draw conclusions about the natural world. This course provides a solid foundation in the principles of computational science, including numerical methods, data analysis, and visualization. It also provides hands-on experience with computational science tools and techniques, which will be invaluable for anyone who wants to pursue a career in computational science.

See salaries and explore the career path for Computational Scientist

Data Scientist

Data Scientists use data to solve business problems. They collect, clean, and analyze data to identify trends and patterns. They also develop and use machine learning models to make predictions and recommendations. This course provides a solid foundation in the principles of data science, including data analysis, machine learning, and visualization. It also provides hands-on experience with data science tools and techniques, which will be invaluable for anyone who wants to pursue a career in data science.

See salaries and explore the career path for Data Scientist

Machine Learning Engineer

Machine Learning Engineers develop and maintain machine learning models. They work with data scientists and other users to understand their machine learning needs and develop models that meet those needs. They also work with hardware and software vendors to ensure that the machine learning models they develop are compatible with the latest technologies. This course provides a solid foundation in the principles of machine learning, including machine learning algorithms, data structures, and performance optimization. It also provides hands-on experience with machine learning languages and tools, which will be invaluable for anyone who wants to pursue a career in machine learning.

See salaries and explore the career path for Machine Learning Engineer

Software Engineer

This course may be useful for Software Engineers who want to develop software for high-performance computing systems. The course provides a solid foundation in the principles of HPC, including parallel programming, performance optimization, and system architecture. It also provides hands-on experience with HPC systems, which will be invaluable for anyone who wants to develop software for HPC.

See salaries and explore the career path for Software Engineer

Computer Scientist

This course may be useful for Computer Scientists who want to specialize in high-performance computing. The course provides a solid foundation in the principles of HPC, including parallel programming, performance optimization, and system architecture. It also provides hands-on experience with HPC systems, which will be invaluable for anyone who wants to pursue a career in HPC.

See salaries and explore the career path for Computer Scientist

Systems Analyst

This course may be useful for Systems Analysts who want to work with high-performance computing systems. The course provides a solid foundation in the principles of HPC, including system administration, performance optimization, and application development. It also provides hands-on experience with HPC systems, which will be invaluable for anyone who wants to work with HPC systems.

See salaries and explore the career path for Systems Analyst

Database Administrator

This course may be useful for Database Administrators who want to work with high-performance computing systems. The course provides a solid foundation in the principles of HPC, including system administration, performance optimization, and application development. It also provides hands-on experience with HPC systems, which will be invaluable for anyone who wants to work with HPC systems.

See salaries and explore the career path for Database Administrator

Network Administrator

This course may be useful for Network Administrators who want to work with high-performance computing systems. The course provides a solid foundation in the principles of HPC, including system administration, performance optimization, and application development. It also provides hands-on experience with HPC systems, which will be invaluable for anyone who wants to work with HPC systems.

See salaries and explore the career path for Network Administrator

Security Analyst

This course may be useful for Security Analysts who want to work with high-performance computing systems. The course provides a solid foundation in the principles of HPC, including system administration, performance optimization, and application development. It also provides hands-on experience with HPC systems, which will be invaluable for anyone who wants to work with HPC systems.

See salaries and explore the career path for Security Analyst

Business Analyst

This course may be useful for Business Analysts who want to understand the potential of high-performance computing for their businesses. The course provides a solid foundation in the principles of HPC, including its benefits, challenges, and applications. It also provides hands-on experience with HPC systems, which will be invaluable for anyone who wants to make informed decisions about using HPC for their businesses.

See salaries and explore the career path for Business Analyst

Project Manager

This course may be useful for Project Managers who are working on projects that involve high-performance computing. The course provides a solid foundation in the principles of HPC, including its benefits, challenges, and applications. It also provides hands-on experience with HPC systems, which will be invaluable for anyone who wants to successfully manage projects that involve HPC.

See salaries and explore the career path for Project Manager

Technical Writer

This course may be useful for Technical Writers who want to write about high-performance computing. The course provides a solid foundation in the principles of HPC, including its benefits, challenges, and applications. It also provides hands-on experience with HPC systems, which will be invaluable for anyone who wants to write clear and accurate documentation about HPC.

See salaries and explore the career path for Technical Writer