Scribd
Upload
171 views21 pages

10.introduction To Data-Parallel Architectures

This document discusses data-parallel architectures. It introduces SIMD (Single Instruction Multiple Data) and describes how one instruction is broadcast to multiple processing elements that each operate on their own data. Connectivity between processing elements is important and can include near-neighbor, tree, pyramid and hypercube topologies. Data-parallel approaches are well-suited for multimedia, image processing, and scientific applications by operating in parallel on large data sets.

Uploaded by

Ashok Ashokbyadav
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
171 views21 pages

10.introduction To Data-Parallel Architectures

This document discusses data-parallel architectures. It introduces SIMD (Single Instruction Multiple Data) and describes how one instruction is broadcast to multiple processing elements that each operate on their own data. Connectivity between processing elements is important and can include near-neighbor, tree, pyramid and hypercube topologies. Data-parallel approaches are well-suited for multimedia, image processing, and scientific applications by operating in parallel on large data sets.

Uploaded by

Ashok Ashokbyadav
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
You are on page 1/ 21

10.

Introduction to Data-Parallel architectures


• SIMD {Single Instruction Multiple Data}
• 10.1 Introduction
• 10.2 Connectivity
• 10.3 Alternative architecture

• e.g. add: (c1=a1+b1), (c2=a2+b2), (c3=a3+b3)

TECH
Computer Science
CH01
Introduction
• Single instruction, multiple data
• One instruction stream is broadcast to all processors
• Each processor, also called a processing element (or
PE), is usually simplistic and logically is essentially
an ALU;
– PEs do not store a copy of the program
nor have a program control unit.
• Individual processors can remain idle during
execution of segments of the program (based on a
data test).
• All active processor executes the same instruction
synchronously, but on different data
Cont…
• Technically, on a memory access, all active
processors must access the same location in their
local memory.
This requirement is sometimes relaxed a bit.
• The data items form an array (or vector) and an
instruction can act on the complete array in one cycle.
• Quinn also calls this architecture a processor array.
Some examples include
ILLIAC IV (1974) was the first SIMD computer
The STARAN and MPP (Dr. Batcher architect)
Connection Machine CM2, built by Thinking
Machines).
MasPar (for Massively Parallel) computers.
Data-parallel computation (bit parallel)
Scalar vs. SIMD
Ex:Multimedia Extension Architectures
• At the core of multimedia extensions
– SIMD parallelism
– Variable-sized data fields:
– Vector length = register width / type size
Application of Data-parallel Architectures:
One data entity processed by one PE
Multimedia / Scientific Applications
• Image
– Graphics : 3D games, movies
– Image recognition
– Video encoding/decoding : JPEG, MPEG4
• Sound
– Encoding/decoding: IP phone, MP3
– Speech recognition
– Digital signal processing: Cell phones
• Scientific applications
– Double precision Matrix-Matrix multiplication
(DGEMM)
– Y[] = a*X[] + Y[] (SAXPY)
Connectivity
• Important aspects of the design space of data parallel
computers is the connectivity.

• This connectivity is established between processing


elements(PEs).

• The inter connectivity methods are:


• Near neighbor, tree, pyramid and hypercube—widely
used data parallel connectivity.

• Bus ,crossbar, and multistage networks –used for


functional parallel design
Mapping Problem space into Architectural Space:
Data entity onto PE (1-to-1 mapping)
Near- Neighbor Connectivity

• It is arose in the context of spatial mapping coherent


data on SIMD systems.
• The data is spatially correlated.
• Ex: 8-connected near neighbor case
Near-neighbor connectivity (2-D: Mesh)
Example of a 2-D Processor Interconnection
Network in a Processor Array

Each VLSI chip has 16 processing elements.


Each PE can simultaneously send a value to a neighbor.

PE =
processor
element

13
Trees and Graphs
• Near-Neighbour Connected system is specialized for
image Processing applications.
• Any general problem has a natural expression in the
form of a graph.
• The problems that are represented as hierarchical
structure like database searching, model matching and
expert system can be represented through a tree
connectivity.
• In this many simultaneous processes are executed at a
low level, then progressively fewer at higher level.
• The computations at this level depends on the results
of lower level.
Tree: 2-D hierarchy
The Pyramid
• The basic architectutre of the pyramid is a for
connected mesh.
• Connection between the layers is fixed and every
processor is connected to one element in the layer
above and four elements in the layer below.
• This approach is suitable for basic image processing
applications.
• Consider the data passing in an N ×N mesh.
• To pass an item of data from one corner of the array
to that diagonally opposite need 2N shift operations.
• If it forms a base level of pyramid ,the same operation
can be performed in 2log2N shifts.
Pyramid: 3-D hierarchy
Hypercube: 2^N nodes in N dimension
Hypercube: 4-D
Long and short-range connections
Data-parallel approaches
Principal characteristics of data-parallel systems

You might also like