May 1, 2024
Updated June 28, 2025
15 minute read
An Introduction to Parallel Computing
Parallel computing is a type of computation where many calculations or processes are carried out simultaneously. Large problems can often be divided into smaller ones, which can then be solved at the same time. This principle is the foundation of parallel computing. It stands in contrast to sequential computing, where a traditional processor executes one instruction at a time. By tackling multiple parts of a problem concurrently, we can achieve results much faster or solve problems of a scale that would be impossible for a single processor to handle in a reasonable amount of time.
3gp6q0|
Find a path to becoming a Parallel Computing. Learn more at:
OpenCourser.com/topic/3gp6q0/parallel
Reading list
We've selected 26 books
that we think will supplement your
learning. Use these to
develop background knowledge, enrich your coursework, and gain a
deeper understanding of the topics covered in
Parallel Computing.
Provides a comprehensive introduction to parallel computing, covering architectures, programming paradigms, algorithms, and standards like MPI and OpenMP. It is an excellent resource for gaining a broad understanding of the field and is often used as a textbook in academic settings.
Focuses on parallel programming for GPUs using CUDA. It provides a hands-on approach with detailed examples and case studies, making it highly relevant for those interested in accelerating applications on many-core architectures.
Offers a hands-on introduction to parallel programming, focusing on key frameworks like MPI, Pthreads, and OpenMP. It is suitable for students and professionals with a background in C programming and provides numerous programming exercises.
This more recent edition focusing on Intel's Threading Building Blocks (TBB) with modern C++ (C++11 and later). It provides updated interfaces and insights based on user experience, making it a current reference for TBB programming. It is suitable for C++ developers interested in leveraging TBB for parallel processing.
Covers the fundamental elements of parallel computing, providing a solid foundation for understanding the principles and techniques involved. It is suitable for students and professionals seeking a comprehensive overview of the field.
Practical guide to using OpenMP for shared-memory parallel programming. It covers the essential concepts and directives of OpenMP, making it a useful resource for implementing parallel programs on multi-core processors. It serves as a good reference for students and practitioners.
Well-regarded resource for understanding the complexities of multiprocessor programming and concurrent data structures. It focuses on the theoretical foundations and practical aspects of designing and implementing concurrent algorithms.
Specifically addresses scaling Python applications using Dask, a flexible library for parallel computing in Python. It is highly relevant for data scientists and machine learning practitioners who need to process large datasets using parallel techniques in Python.
Considered a classic in computer architecture, this book provides foundational knowledge on how computer systems are designed, including principles of parallelism and memory hierarchy. While not solely focused on parallel computing, it offers essential context for understanding the underlying hardware.
Provides an introduction to high-performance computing with a focus on scientific applications. It covers topics relevant to parallel computing in this domain, such as performance optimization and using parallel programming models in scientific contexts.
Is part of a series that explores various approaches to multicore and many-core programming. It offers insights into high-performance parallelism through a collection of articles and examples, providing practical techniques for optimizing parallel code.
This book, available online, covers the concepts and experiences of designing and building parallel programs. It provides a practical perspective on the challenges and techniques involved in developing parallel software. While not a recent publication, it offers valuable insights into the process.
Focuses on achieving high performance in Python for large datasets, including techniques related to parallelism. It is relevant for those working with Python and data-intensive applications, offering practical methods for improving performance through parallel approaches.
Explores parallel and concurrent programming techniques using the Haskell language. It provides a functional programming perspective on parallelism, which can be valuable for understanding alternative approaches to concurrent and parallel problem-solving. It is suited for those with a functional programming background or an interest in exploring different paradigms.
Aims to provide an intuitive understanding of concurrency, a foundational concept for parallel computing. While not solely focused on parallelism, a strong grasp of concurrency is crucial for effective parallel programming. It is likely more conceptual and introductory.
Provides a comprehensive overview of parallel computing, covering topics such as parallel architectures, algorithms, and applications. It is written by an expert in the field and is suitable for both undergraduate and graduate students.
Provides a comprehensive overview of parallel computing for scientific and engineering applications, covering topics such as parallel programming models, algorithms, and performance optimization. It is written by a team of experts in the field and is suitable for both researchers and practitioners.
Provides a practical guide to parallel computing, covering topics such as parallel programming models, algorithms, and performance optimization. It is written by a team of experts in the field and is suitable for both programmers and researchers.
Provides a comprehensive overview of parallelism in OpenMP, covering topics such as parallel programming models, algorithms, and performance optimization. It is written by an expert in the field and is suitable for both programmers and researchers.
For more information about how these books relate to this course, visit:
OpenCourser.com/topic/3gp6q0/parallel
Save
