Dichotomy of parallel computing platforms
Download as PPTX, PDF4 likes7,341 views
The document discusses the dichotomy between SIMD (Single Instruction stream, Multiple Data stream) and MIMD (Multiple Instruction stream, Multiple Data stream) parallel computing platforms, focusing on their control structures and processing units. SIMD processors execute the same instruction across multiple processors, often requiring an activity mask for selective computation, while MIMD processors can handle different instructions on different data. The comparison highlights that SIMD systems require less hardware but are costlier and have longer design cycles, whereas MIMD platforms can be constructed from cheaper, off-the-shelf components.
Dichotomy of parallel computing platforms
- 1. Dichotomy of Parallel Computing Platforms • An explicitly parallel program must specify concurrency and interaction between concurrent subtasks. • The former is sometimes also referred to as the control structure and the latter as the communication model.
- 2. Control Structure of Parallel Programs • Parallelism can be expressed at various levels of granularity - from instruction level to processes. • Between these extremes exist a range of models, along with corresponding architectural support.
- 3. Control Structure of Parallel Programs • Processing units in parallel computers either operate under the centralized control of a single control unit or work independently. • If there is a single control unit that dispatches the same instruction to various processors (that work on different data), the model is referred to as single instruction stream, multiple data stream (SIMD). • If each processor has its own control control unit, each processor can execute different instructions on different data items. This model is called multiple instruction stream, multiple data stream (MIMD).
- 4. SIMD and MIMD Processors A typical SIMD architecture (a) and a typical MIMD architecture (b).
- 5. SIMD Processors • Some of the earliest parallel computers such as the Illiac IV, MPP, DAP, CM-2, and MasPar MP-1 belonged to this class of machines. • Variants of this concept have found use in co-processing units such as the MMX units in Intel processors and DSP chips such as the Sharc. • SIMD relies on the regular structure of computations (such as those in image processing). • It is often necessary to selectively turn off operations on certain data items. For this reason, most SIMD programming paradigms allow for an ``activity mask'', which determines if a processor should participate in a computation or not.
- 6. Conditional Execution in SIMD Processors Executing a conditional statement on an SIMD computer with four processors: (a) the conditional statement; (b) the execution of the statement in two steps.
- 7. MIMD Processors • In contrast to SIMD processors, MIMD processors can execute different programs on different processors. • A variant of this, called single program multiple data streams (SPMD) executes the same program on different processors. • It is easy to see that SPMD and MIMD are closely related in terms of programming flexibility and underlying architectural support. • Examples of such platforms include current generation Sun Ultra Servers, SGI Origin Servers, multiprocessor PCs, workstation clusters, and the IBM SP.
- 8. SIMD-MIMD Comparison • SIMD computers require less hardware than MIMD computers (single control unit). • However, since SIMD processors ae specially designed, they tend to be expensive and have long design cycles. • Not all applications are naturally suited to SIMD processors. • In contrast, platforms supporting the SPMD paradigm can be built from inexpensive off-the-shelf components with relatively little effort in a short amount of time.