Scribd
Upload
53 views

Parallel Algo C Sar

The document outlines an agenda for a course on parallel algorithms, discussing topics like Flynn's taxonomy of computer architectures, parallel programming languages, precedence graphs, elementary parallel algorithms for tasks like summation and sorting, and algorithms for parallel matrix multiplication and sorting networks. Examples are provided to illustrate parallel algorithms for summation, broadcast, odd-even transposition sort, and systolic matrix multiplication on different architectures.

Uploaded by

Evelyn Dias Santos
Copyright
© © All Rights Reserved
Available Formats
Download as PPSX, PDF, TXT or read online on Scribd
53 views

Parallel Algo C Sar

The document outlines an agenda for a course on parallel algorithms, discussing topics like Flynn's taxonomy of computer architectures, parallel programming languages, precedence graphs, elementary parallel algorithms for tasks like summation and sorting, and algorithms for parallel matrix multiplication and sorting networks. Examples are provided to illustrate parallel algorithms for summation, broadcast, odd-even transposition sort, and systolic matrix multiplication on different architectures.

Uploaded by

Evelyn Dias Santos
Copyright
© © All Rights Reserved
Available Formats
Download as PPSX, PDF, TXT or read online on Scribd
You are on page 1/ 28

2010 - 02 -17

CS 515: Parallel Algorithms


Chandrima Sarkar Atanu Roy

Agenda
Architecture Parallel Programming Languages Precedence Graph Elementary Parallel Algorithms Sorting Matrix Multiplication Download :http://www.cs.montana.edu/~atanu.roy/Classes/ CS515.html

Architecture
Flynns Classification S = single , M = multiple , I = instruction (stream), D = data (stream)

SISD

SIMD

Architecture
Flynns Classification S = single , M = multiple , I = instruction (stream), D = data (stream)

MISD

MIMD

Static Inter-connection Network


Linear Array Ring

Ring arranged to use short wires

Fully Connected Topology

Chordal ring

Multidimensional Meshes and Torus

Tree

Tree Cont.

FAT TREE

STAR

Hypercube
100 000 110

010
101 111

001 0-D 1-D 2-D

011 3-D 4-D

5-D

Parallel Programming Languages


Control Mechanism Communication Mechanism

Shared Memory Control driven

Message-passing

Fortran 90/HPF , C++ , HEP CSP , Ada , OCCAM (Von PL/I , Ada , Concurrent Neumann Language Pascal Modula-2 , MultiLisp Extension ) (MIMD), Lisp Connection Machine (SIMD) VAL , ID LAU , SISAL ( dataflow languages )
Concurrent Prolog ( Shapiro ) Actors FP

Data driven
Pattern driven Demand driven ( reduction language )

Dijkstras High Level language construct


Degree of Parallelism is static Algol-68,CSP Precendence Graph

A
parbegin C begin B parbegin D E parend G end parend H

Elementary Parallel Algorithms


Finding sum using a 2D mesh architecture

Finding sum of 16 values in a Shuffle Exchange SIMD Model

Parallel summation in a Hypercube SIMD Model

Broadcast in a Hypercube
Algorithm 1

Algorithm 2

Odd Even Transposition Sort


(1) p = n

14 5 15 8 4 11 13 12 odd-even 14 5 15 8 4 11 13 12 even-odd 14 5 15 4 8 11 13 12 odd-even 5 14 4 15 8 11 12 13 even-odd 5 4 14 8 15 11 12 13 odd-even 4 5 8 14 11 15 12 13 even-odd 4 5 8 11 14 12 15 13 odd-even 4 5 8 11 12 14 13 15 even-odd 4 5 8 11 12 13 14 15

Odd Even Transposition Sort (contd)


(2) p << n
S= {12, 7, 2, 4, 1, 11, 9, 5, 6, 3, 10, 8}, p = 4
P1 {12, 7, 2} {2, 7, 12} {1, 2, 4} {1, 2, 4} {1, 2, 3} {1, 2, 3} P2 {4, 1, 11} {1, 4, 11} {7, 11, 12} {3, 5, 6} {4, 5, 6} {4, 5, 6} P3 {9, 5, 6} {5, 6, 9} {3, 5, 6} {7, 11, 12} {7, 8, 9} {7, 8, 9} P4 {3, 10, 8} {3, 8, 10} {8, 9, 10} {8, 9, 10} {10, 11, 12} {10, 11, 12}

Pseudocode
Proc MERGE-SPLIT(S)
for i:= 1 to p do in parallel
QUICKSORT(Si)

end for for (i := 1 to ceil(p/2)) for odd-numbered processor do in parallel MERGE(Si , Si + 1) SPLIT end for for odd-numbered processor do in parallel MERGE(Si , Si + 1) SPLIT end for end for

2 D mesh with Snake Order


Thompson and Kung (1977)

Input : {23, 6, 1, 5, 11, 13, 55, 19, -3, 12, -5, -7, 9, 55, 28, -2}

Snake Order (contd.)

Bitonic Merge Sort


Bitonic Sequence :- 1, 3, 7, 8 Comparator Note :- Batchers Bitonic Merge Sort compares elements whose indices differ by a single bit. 6, 5, 4, 2

Bitonic Merge Sort

Shuffle-Exchange Network
Bitonic Mergesort on Shuffle-Exchange Network A list of n = 2k unsorted elements can be sorted in time (lg2 n) with a network 2k-1[k (k-1) + 1] comparators using the shuffle-exchange network.

Sorting Network

Odd Even Merging Network

Systolic Matrix Multiplication


1. 2. 3. 4. Multiply ai,k by ak,j Add the result to ri,j Send ai,k to cell ci+1,j Send bk,j to cell ci,j+1

Home Work
Show how the following 16 values would be sorted by Batchers Bitonic sort. 16, 7, 4, 12, 2, 10, 13, 9, 1, 8, 11, 3, 15, 6, 5, 14

You might also like