Faculty of Engineering Savitribai Phule Pune University

Savitribai Phule Pune University

Fourth Year of Computer Engineering (2019 Course)
410250: High Performance Computing
Teaching Scheme: Credit Examination Scheme:
TH: 3 Hours/Week 3 In- Sem (TH) : 30
End- Sem (TH): 70
Prerequisites Courses: -Microprocessor (210254), Principles of Programming
Languages(210255), Computer Networks and Security(310244)
Companion Course: Laboratory Practice V(410254)
Course Objectives:
 To understand different parallel programming models
 To analyze the performance and modeling of parallel programs
 To illustrate the various techniques to parallelize the algorithm
 To implement parallel communication operations.
 To discriminate CUDA Architecture and its components.
 To Understand Scope of Parallel Computing and its search algorithms.
Course Outcomes:
CO1: Understand various Parallel Paradigm
CO2: Design and Develop an efficient parallel algorithm to solve given problem
CO3: Illustrate data communication operations on various parallel architecture
CO4: Analyze and measure performance of modern parallel computing systems
CO5: Apply CUDA architecture for parallel programming
CO6: Analyze the performance of HPC applications

Course Contents
Unit I Introduction to Parallel Computing 09 Hours
Introduction to Parallel Computing: Motivating Parallelism, Modern Processor: Stored- program
computer architecture, General-purpose Cache-based Microprocessor architecture. Parallel
Programming Platforms: Implicit Parallelism, Dichotomy of Parallel Computing Platforms, Physical
Organization of Parallel Platforms, Communication Costs in Parallel Machines. Levels of parallelism,
Models: SIMD, MIMD, SIMT, SPMD, Data Flow Models, Demand-driven Computation, Architectures:
N-wide superscalar architectures, multi-core, multi-threaded.
#Exemplar/Case
Studies Case study: Multi-core System
*Mapping of Course
Outcomes for Unit I CO1
Unit II Parallel Algorithm Design 09 Hours
Principles of Parallel Algorithm Design: Preliminaries, Decomposition Techniques, Characteristics of
Tasks and Interactions, Mapping Techniques for Load Balancing, Methods for Containing Interaction
Overheads, Parallel Algorithm Models: Data, Task, Work Pool and Master Slave Model, Complexities:
Sequential and Parallel Computational Complexity, Anomalies in Parallel Algorithms.
#Exemplar/Case Foster's parallel algorithm design methodology.
Studies (http://compsci.hunter.cuny.edu/~sweiss/course_materials/csci493.65/lecture_n

Syllabus for Fourth Year of Computer Engineering #72/4

 Faculty of Engineering Savitribai Phule Pune University
otes/chapter03.pdf)

*Mapping of Course
Outcomes for Unit II CO2
Unit III Parallel Communication 09 Hours
Basic Communication: One-to-All Broadcast, All-to-One Reduction, All-to-All Broadcast and
Reduction, All-Reduce and Prefix-Sum Operations, Collective Communication using MPI: Scatter,
Gather, Broadcast, Blocking and non blocking MPI, All-to-All Personalized Communication, Circular
Shift, Improving the speed of some communication operations.
#Exemplar/Case Case study: Monte-Carlo Pi computing using MPI
Studies
*Mapping of Course CO3
Outcomes for UnitIII

Unit IV Analytical Modeling of Parallel Programs 09 Hours

Sources of Overhead in Parallel Programs, Performance Measures and Analysis: Amdahl's and
Gustafson's Laws, Speedup Factor and Efficiency, Cost and Utilization, Execution Rate and Redundancy,
The Effect of Granularity on Performance, Scalability of Parallel Systems, Minimum Execution Time
and Minimum Cost, Optimal Execution Time, Asymptotic Analysis of Parallel Programs. Matrix
Computation: Matrix-Vector Multiplication, Matrix-Matrix
Multiplication.
#Exemplar/Case Case study: The DAG Model of parallel computation
Studies
*Mapping of Course
Outcomes for UnitIV CO4

Unit V CUDA Architecture 09 Hours

Introduction to GPU: Introduction to GPU Architecture overview, Introduction to CUDA C- CUDA
programming model, write and launch a CUDA kernel, Handling Errors, CUDA memory model, Manage
communication and synchronization, Parallel programming in CUDA- C.
#Exemplar/Case Case study: GPU applications using SYCL and CUDA on NVIDIA
Studies
*Mapping of Course
Outcomes for Unit V CO5
Unit VI High Performance Computing Applications 09 Hours

Scope of Parallel Computing, Parallel Search Algorithms: Depth First Search(DFS), Breadth First
Search( BFS), Parallel Sorting: Bubble and Merge, Distributed Computing: Document classification,
Frameworks – Kuberbets, GPU Applications, Parallel Computing for AI/ ML
#Exemplar/Case Case study: Disaster detection and management/ Smart Mobility/Urban
Studies planning
*Mapping of Course CO6
Outcomes for Unit
VI
Syllabus for Fourth Year of Computer Engineering #73/4
 Faculty of Engineering Savitribai Phule Pune University
Learning Resources
Text Books:
1. Ananth Grama, Anshul Gupta, George Karypis, and Vipin Kumar, "Introduction toParallel
Computing", 2nd edition, Addison-Wesley, 2003, ISBN: 0-201-64865-2
2. Seyed H. Roosta, ―Parallel Processing and Parallel Algorithms Theory and Computation‖‖,
Springer-Verlag 2000 ,ISBN 978-1-4612-7048-5 ISBN 978-1-4612-1220-1
3. John Cheng, Max Grossman, and Ty McKercher, ―Professional CUDA C Programming‖, John
Wiley & Sons, Inc., ISBN: 978-1-118-73932-7

Reference Books :
1. Kai Hwang,, "Scalable Parallel Computing", McGraw Hill 1998.
2. George S. Almasi and Alan Gottlieb, "Highly Parallel Computing", The Benjamin and
Cummings Pub. Co., Inc
3. Jason sanders, Edward Kandrot, ― CUDA by Example‖, Addison-Wesley, ISBN-13: 978- 0-13-
138768-3
4. Pacheco, Peter S., ―An Introduction to Parallel Programming‖, Morgan Kaufmann
Publishers ISBN 978-0-12-374260-5
5. Rieffel WH.EG, Polak, ― Quantum Computing: A gentle introduction‖, MIT Press, 2011,
ISBN 978-0-262-01506-6
6. Ajay D. Kshemkalyani , Mukesh Singhal, ― Distributed Computing: Principles, Algorithms,
andSystems‖, Cambridge March 2011, ISBN: 9780521189842
e Books :
1. http://prdrklaina.weebly.com/uploads/5/7/7/3/5773421/introduction_to_high_performance_computi
ng_for_scientists_and_engineers.pdf

2. https://www.vssut.ac.in/lecture_notes/lecture1428643084.pdf

NPTEL/YouTube video lecture link

● https://nptel.ac.in/courses/106108055
● https://www.digimat.in/nptel/courses/video/106104120/L01.html

@The CO-PO Mapping Matrix

CO/
PO1 PO2 PO3 PO4 PO5 PO6 PO7 PO8 PO9 PO10 PO11 PO12
PO
CO1 2 1 - - - - - - - - - -
CO2 2 1 - - - - - - - - - -
CO3 2 1 - - - - - - - - - -
CO4 1 2 - 2 - - - - - - - -
CO5 1 2 - 2 - - - - - - - 1
CO6 2 2 - 2 - - - - - - - 1

Syllabus for Fourth Year of Computer Engineering #74/4

 Faculty of Engineering Savitribai Phule Pune University

Syllabus for Fourth Year of Computer Engineering #75/4