Scribd
Upload
5 views

PP - CH03

The document discusses parallel programming, focusing on writing parallel algorithms and architectural considerations such as shared and distributed memory systems. It introduces distributed shared memory (DSM) and various algorithms for its implementation, including the Central Server Algorithm. Additionally, it covers the identification of parallelism, time complexity, asymptotic analysis, and SPMD programming as a method for achieving parallelism.

Uploaded by

Ahmed Sayed
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
5 views

PP - CH03

The document discusses parallel programming, focusing on writing parallel algorithms and architectural considerations such as shared and distributed memory systems. It introduces distributed shared memory (DSM) and various algorithms for its implementation, including the Central Server Algorithm. Additionally, it covers the identification of parallelism, time complexity, asymptotic analysis, and SPMD programming as a method for achieving parallelism.

Uploaded by

Ahmed Sayed
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 15

Parallel Programming

Considerations
CHAPTER 03 : PART 01
Agenda

 Writing parallel programs


 Architectural Considerations
 Distributed Memory
Writing parallel programs

 An algorithm is a sequence of steps that take inputs from the user and after
some computation, produces an output. A parallel algorithm is an algorithm
that can execute several instructions simultaneously on different processing
devices and then combine all the individual outputs to produce the final
result.
 concurrent processing : that can divide a complex task and process it
multiple systems to produce the output in quick time.
Architectural Considerations

 Shared-memory machines, which have a single shared address space that


can be accessed by any processor.

 Distributed-memory machines, in which the system memory is packaged


with individual nodes of one or more processors and communication is
required to provide data from the memory of one processor to a different
processor.
Distributed Memory

 Distributed Shared Memory abbreviated as


DSM is the implementation of shared
memory concept in distributed systems.
 The DSM system implements the shared
memory models in loosely coupled systems
that are deprived of a local physical shared
memory in the system. In this type of
system distributed shared memory provides
a virtual memory space that is accessible
by all the system (also known as nodes) of
the distributed hierarchy.
Algorithm for implementing
Distributed Shared Memory

 Based on these challenges there are algorithms designed to implement


distributed shared memory. There are four algorithms −
 Central Server Algorithm
 Migration Algorithm

 Read Replication Algorithm


 Full Replication Algorithm
Central Server Algorithm

 All shared data is maintained by the central server. Other nodes


of the distributed system request for reading and writing data to
the server which serves the request and updates or provides
access to the data along with acknowledgment messages.
 These acknowledgment messages are used to provide the status
of the data request is served by the server. When the data is
sent to the calling function, it acknowledges a number that
shows the access sequence of the data to maintain concurrency.
And time-out is returned in case of failure.
 For larger distributed systems, there can be more than one
server. In this case, the servers are located using their address
or using mapping functions.
Parallel Programming
Considerations
CHAPTER 03 : PART 02
Agenda

Identification of Parallelism
Parallel Algorithms

Asymptotic Analysis

SPMD Programming
Identification of Parallelism

 Analysis of an algorithm helps us determine whether the algorithm is useful or not.


Generally, an algorithm is analyzed based on its execution time (Time Complexity) and
the amount of space (Space Complexity) it requires.
 Since we have sophisticated memory devices available at reasonable cost, storage space
is no longer an issue. Hence, space complexity is not given so much of importance.
 Parallel algorithms are designed to improve the computation speed of a computer. For
analyzing a Parallel Algorithm, we normally consider the following parameters −
 Time complexity (Execution Time),
 Total number of processors used, and
 Total cost.
Time Complexity

 The main reason behind developing parallel algorithms was to reduce the computation time of an algorithm.
Thus, evaluating the execution time of an algorithm is extremely important in analyzing its efficiency.
 Execution time is measured on the basis of the time taken by the algorithm to solve a problem. The total
execution time is calculated from the moment when the algorithm starts executing to the moment it stops. If all
the processors do not start or end execution at the same time, then the total execution time of the algorithm is
the moment when the first processor started its execution to the moment when the last processor stops its
execution.
 Time complexity of an algorithm can be classified into three categories−
 Worst-case complexity − When the amount of time required by an algorithm for a given input is
maximum.
 Average-case complexity − When the amount of time required by an algorithm for a given input is
average.
 Best-case complexity − When the amount of time required by an algorithm for a given input is minimum.
Asymptotic Analysis

 The complexity or efficiency of an algorithm is the number of steps executed by the


algorithm to get the desired output. Asymptotic analysis is done to calculate the
complexity of an algorithm in its theoretical analysis. In asymptotic analysis, a large
length of input is used to calculate the complexity function of the algorithm.
 Asymptotic notation is the easiest way to describe the fastest and slowest possible
execution time for an algorithm using high bounds and low bounds on speed. For this, we
use the following notations −
 Big O notation : f(n) = O(g(n))
 Omega notation : f(n) = Ω (g(n))
 Theta notation : f(n) = θ(g(n))
Ο Notation “upper bound” the worst case time complexity
Ω Notation “lower bound” the best case time complexity
θ Notation “lower bound and the upper bound”
Speedup of an Algorithm 01

 The performance of a parallel algorithm is determined by calculating its speedup.


Speedup is defined as the ratio of the worst-case execution time of the fastest known
sequential algorithm for a particular problem to the worst-case execution time of the
parallel algorithm.

Speedup =
Speedup of an Algorithm 02

 The number of processors used is an important factor in analyzing the efficiency


of a parallel algorithm. The cost to buy, maintain, and run the computers are
calculated. Larger the number of processors used by an algorithm to solve a
problem, more costly becomes the obtained result.
 Total cost of a parallel algorithm is the product of time complexity and the
number of processors used in that particular algorithm.
 Total Cost = Time complexity × Number of processors used
 Therefore, the efficiency of a parallel algorithm is −
 Efficiency =
SPMD Programming

 SPMD (single program, multiple data) is a technique employed to achieve


parallelism; it is a subcategory of MIMD.
 Tasks are split up and run simultaneously on multiple processors with
different input in order to obtain results faster.
 SPMD is the most common style of parallel programming.
 It is also a prerequisite for research concepts such as active messages and
distributed shared memory.

You might also like