Parallel & Distributed

Computing

By: M. Imran Siddiqui

Parallel Computing
SISD architecture is suitable for tasks that can be efficiently executed in a sequential
manner, where the focus is on processing one instruction at a time on a single piece of
data. It is the most straightforward and common architecture, especially in everyday
computing devices.
examples
Most personal computers, such as those running x86 or x86-64 architectures, operate
as SISD systems. The CPU processes instructions sequentially on a single set of data.
Many embedded systems, such as those found in appliances, automotive control
systems, and industrial machines, are based on SISD architecture.
Simple microcontrollers with a single processing core often follow the SISD architecture.
• SIMD architecture is designed to process multiple data elements simultaneously using a single instruction. This type
of architecture is well-suited for parallel processing tasks where the same operation needs to be performed on a
large set of data. Here are some examples of SIMD architectures and technologies:
• the broad application of SIMD architectures in various computing domains, including scientific simulations,
graphics, signal processing, and general-purpose parallel computing. Keep in mind that SIMD is just one of the
categories in Flynn's taxonomy, and different architectures may be better suited to specific types of applications.
• Vector Processors:
– Cray-1: The Cray-1 supercomputer, introduced in the 1970s, was one of the first machines to use vector processing for
scientific and engineering simulations.
• Graphics Processing Units (GPUs):
– NVIDIA CUDA and AMD Stream: Modern GPUs are powerful SIMD devices that excel at parallel processing. They are
extensively used in graphics rendering as well as general-purpose computing tasks.
• Digital Signal Processors (DSPs):
– TI C6000 series: Texas Instruments' C6000 series of DSPs use SIMD instructions for efficient signal processing applications.
• Intel SIMD Extensions (SSE) and Advanced Vector Extensions (AVX):
– SSE and AVX: These are SIMD instruction sets developed by Intel for their x86 and x86-64 architectures. SSE and AVX
instructions enable parallel processing of data in multimedia and scientific applications.
it is not commonly implemented in practical
systems. In MISD architectures, multiple
processors or processing units operate on a
single data stream using different instruction
streams. While this classification exists in
theory, real-world examples are rare because
it is challenging to design systems that
effectively utilize multiple instruction streams
on a single data stream.
MIMD (Multiple Instruction streams, Multiple Data streams) architectures are prevalent in parallel computing
systems, where multiple processors or processing elements operate independently on different data and
instructions. Here are some examples of MIMD architectures:
1.Multi-Core Processors:
• Modern CPUs often have multiple cores, each capable of executing its own instruction stream. Processes and
threads can run concurrently on these cores, providing parallel processing capabilities.
2.Clustered Systems:
• Clustered systems consist of multiple independent computers (nodes) connected together. Each node in the
cluster operates independently and has its own memory and processing resources. Clusters are commonly
used in high-performance computing (HPC) applications.
3.Distributed Computing Systems:
• In distributed computing, tasks are divided among multiple independent computers connected through a
network. Each computer runs its own set of instructions on its local data. Examples include MapReduce
frameworks like Apache Hadoop.