Scribd
Upload
3 views5 pages

One

The document outlines a course on Parallel and Distributed Computing offered by the Department of Computer Engineering, detailing its objectives, learning outcomes, and topics covered. Students will learn about the principles, architectures, algorithms, and performance optimization techniques related to distributed and parallel computing. Assessment methods include theory assignments, presentations, and in-class tests, with reference books provided for further reading.

Uploaded by

malvinrobotics
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
3 views5 pages

One

The document outlines a course on Parallel and Distributed Computing offered by the Department of Computer Engineering, detailing its objectives, learning outcomes, and topics covered. Students will learn about the principles, architectures, algorithms, and performance optimization techniques related to distributed and parallel computing. Assessment methods include theory assignments, presentations, and in-class tests, with reference books provided for further reading.

Uploaded by

malvinrobotics
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 5

Faculty of Computer Engineering Informatics

and Communications

Department of Computer Engineering


Module: Parallel and Distributed Computing
Module Code: HCE/HHE/HSE 311
Lecturers: Mr. I Chikukuza/Eng. A. Mukwazvure

Email Address: [email protected]/[email protected]

Mobile No: 0719311812/0777740504

Objectives:
This course will provide students with practical and theoretical knowledge
Parallel and Distributed Computing. On completion of this course, students
should be able to:
 Students should gain an understanding of the principles, concepts, and
challenges of distributed computing.
 Students should learn about parallel processing architectures, parallel
algorithms, and techniques for exploiting parallelism in computation.
 Students should learn techniques for optimizing the performance of
distributed and parallel applications.
 Students should understand the challenges of fault tolerance in
distributed and parallel systems.
 Students should understand the concepts of scalability and resource
management in distributed and parallel computing.
 Students should stay updated with the latest trends and technologies in
distributed and parallel computing, such as cloud computing, edge
computing, and big data processing frameworks.
Learning Outcome:
At the end of the course, the students will be able to do following:

 Students should acquire a solid understanding of the principles,


concepts, and theories related to distributed and parallel computing.
 Students should be able to analyze complex computational problems
and determine whether they can benefit from distributed and parallel
computing approaches.
 Students should learn techniques for optimizing the performance of
distributed and parallel applications.
 Students should develop skills in collaborating with others,
understanding and respecting diverse perspectives, and effectively
communicating and coordinating tasks in distributed and parallel
computing projects.
 Students should develop critical thinking skills to evaluate the
advantages, limitations, and trade-offs of different distributed and
parallel computing approaches.
 Students should be aware of the ethical implications and considerations
associated with distributed and parallel computing
 Students should develop a mindset of continuous learning and stay
updated with the latest advancements and technologies in distributed
and parallel computing.

Topic Content
Introduction to Parallel and  Overview of parallel and distributed computing
Distributed Computing concepts
 Parallel and distributed computing strategies
 Benefits of parallel and distributed computing
 Fundamental differences between parallel and
distributed computing
 Examples of real-world applications
Parallel Computing Architectures  Flynn's taxonomy: SISD, SIMD, MISD, MIMD
 Shared memory architectures (e.g., multicore
processors)
 Distributed memory architectures (e.g.,
computer clusters)
 Hybrid architectures (e.g., GPUs)
 Hardware considerations in parallel computing.
 Scalability and performance considerations
Models of Computation  Overview of parallel computation models
(PRAM, BSP).
 Distributed system models (client-server, peer-
to-peer).
 Characteristics of parallelism: data, task, and
pipeline.

Distributed Systems  Characteristics and challenges of distributed


Fundamentals systems.
 Communication models: message passing vs.
shared memory.
 Synchronization and coordination in distributed
environments.

Parallel Algorithm Design  Design strategies: divide and conquer, dynamic


programming.
 Case studies of classic parallel algorithms (e.g.,
parallel sorting, matrix operations).
 Complexity analysis and performance
evaluation.

Programming Models and Tools  Overview of programming models (OpenMP,


MPI, CUDA).
 Introduction to high-level frameworks (Apache
Spark, Hadoop).
 Practical exercises in parallel programming.

Distributed Algorithms  Consensus and coordination algorithms (Paxos,


Raft).
 Distributed mutual exclusion and leader
election.
 Analysis of distributed algorithms’
performance.

Distributed Computing  Definition of terms


Environments, Coordination and  Key Characteristics of Distributed Computing
Synchronization Environments
 Types of Distributed Computing Environments
 Key Components of a Distributed Computing
Environments
 Challenges of a Distributed Computing
Environments
 Coordination Techniques
 Synchronization Techniques
Performance Evaluation and  Metrics for evaluating performance (speedup,
Scalability efficiency).
 Amdahl’s and Gustafson’s Laws.
 Identifying and addressing performance
bottlenecks.

Fault Tolerance and Reliability  Techniques for fault detection and recovery.
 Replication and consistency models in
distributed systems.
 Case studies on fault tolerance strategies.

High-Performance Computing  What is High-Performance Computing (HPC)


 Key Features of HPC
 Applications of HPC
 Challenges in HPC
Supercomputer  What is a supercomputer
 Key Features
 Applications
 Notable Examples
 Challenges
High Performance Cluster  What it is High Performance Cluster
 Key Features
 Applications
 Advantages
 Challenges

Quantum Computing  Quantum Computing


 Key Concepts
 Applications
 Advantages
 Challenges
Distributed Computing  What is big data analytics
Applications in Big Data Analytics  Key Concepts
 Applications in Big Data Analytics
 Advantages
 Challenges
Emerging Trends in Parallel and  Cloud computing and virtualization
Distributed Computing  Edge and fog computing
 Internet of Things (IoT) and sensor networks
 High-performance computing (HPC) clusters
and supercomputers
 Quantum computing and its implications
Reference Books
 Distributed Systems: Concepts and Design" by George Coulouris, Jean Dollimore,
and Tim Kindberg (2011)
 Parallel Computing: Theory and Practice" by Michael J. Quinn (2018)
 High Performance Computing" by Charles Severance and Kevin Dowd (2018)
 Big Data: Principles and best practices of scalable real-time data systems" by
Nathan Marz and James Warren (2015)

Assessment Methods

Coursework

 Theory Assignments
 Presentations
 In-class tests

You might also like