Parallel processing refers to the simultaneous execution of multiple tasks to enhance computer performance by dividing larger problems into smaller, manageable ones. It encompasses various types such as bit level, data, and task parallelism, and utilizes techniques like pipelining and vector processing. Flynn's Taxonomy classifies parallel processing systems based on instruction and data streams, leading to categories such as SISD, SIMD, MISD, and MIMD.
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0 ratings0% found this document useful (0 votes)
2 views31 pages
COA unit -4
Parallel processing refers to the simultaneous execution of multiple tasks to enhance computer performance by dividing larger problems into smaller, manageable ones. It encompasses various types such as bit level, data, and task parallelism, and utilizes techniques like pipelining and vector processing. Flynn's Taxonomy classifies parallel processing systems based on instruction and data streams, leading to categories such as SISD, SIMD, MISD, and MIMD.
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 31
Parallel processing
• Parallel computing means doing several task
simultaneously, thus improving the performance of the computer system • Parallel processing is a type of computing where numerous instruction are implemented at same time on the principle that larger problems may be divided into smaller ones that are solved in parallel • If a computer system provide the technique for simultaneous processing of different set of data , the compu te r s y s te m i s s a i d to be u s i n g a parallel processing technique Parallel processing • The different type of parallel computing are ⮚ Bit level parallelism ⮚ Data parallelism ⮚ Task parallelism The various technique by which parallel processing can be achieved are as ⮚ pipelining ⮚ Vector processing ⮚ Parallel processors Parallel processing ⮚ Parallel processing is a term used to denote a large class of techniques that are used to provide simultaneous data processing task for the purpose of increasing computational speed of computer system ⮚ Instead of processing each instruction sequentially as in a conventional computer, a parallel processing system is able to perform concurrent data processing to achieve faster execution time ⮚ For example, while an instruction is being executed in the ALU , the next instruction can be read from memory ⮚ The system may have two or more ALU and be able to execute two or more instructions at the same time. ⮚ Furthermore the system may have two or more processors operating concurrently ⮚ The purpose of parallel processing is to speed up the computer processing capability and increase the throughput, that is the amount of processing that can be accomplished during a given interval in time ⮚ The amount of hardware increase with parallel processing, and with it , the cost of the system increases Parallel processing ⮚ Parallel processing can be viewed from various levels of complexity. At the lowest level , we distinguish between parallel and serial operations by the type of registers used. ⮚ Shift register operate in serial fashion one bit at a time while registers with parallel load operate with all the bits of the word simultaneously ⮚ Parallel processing at a higher level of complexity can be achieved by having a multiplicity of functional units that perform identical or different operations simultaneously ⮚ Parallel processing is established by distributing the data among the multiple functional units ⮚ For example the arithmetic logic and shift operations can be separated into three units and the operands diverted to each units under the supervision of control unit Parallel processing ⮚ The possible way of separating the execution unit into eight functional units operating in parallel. Parallel processing ⮚ The operands in the registers are applied to one of the units depending on the operation specified by the instruction associated with the operands ⮚ The operation performed in each functional unit is indicated in each block of the diagram ⮚ The adder and integer multiplier perform the arithmetic operations with integer numbers ⮚ The floating point operations are separated into three circuits operating in parallel ⮚ The logic shift and increment operations can be performed concurrently on different data ⮚ All units are independent of each other , so one number can be shifted while another number is being incremented ⮚ A multifunctional organization is usually associated with a complex control unit to coordinate all the activities among the various components Parallel processing ⮚ There are variety of ways that parallel processing can be classified ⮚ It can be considered from the internal organization of the processors from the interconnection structure between processors, or from the flow of information through the system ⮚ One classification introduced by M J Flyn considers the organization of a computer system by the number of instructions and data items that are manipulated simultaneously ⮚ The sequence of instructions read from memory constitutes an instruction stream ⮚ The operation performed on the data in the processor constitutes a data stream ⮚ Four major classification are Multiple Processor Organization • Single instruction stream, single data stream - SISD • Single instruction stream, multiple data stream - SIMD • Multiple instruction stream, single data stream - MISD • Multiple instruction stream, multiple data stream- MIMD Flynn’s Taxonomy
⮚ Flynn’s Taxonomy uses two basic concepts:
Parallelism in instruction stream, and parallelism in data stream. ⮚ A n CPU system has n program counter, so there are n “instruction stream” that can execute in parallel. ⮚ A data stream can be used as a sequence of data, and there exist 4 possible combinations. Flynn’s Classification Flynn’s Classification
Insrtuction Data Name Examples
Set Streams 1 1 SISD Von Neumann Machine 1 Multiple SIMD Vector Super Computer Multiple 1 MISD Arguably None
SISD(Single Instruction Single Data) ⮚ A processor that can only do one job at a time from start to finish. Single Instruction, Single Data Stream - SISD • SISD represent the organization of a single computer containing a control unit, a processor unit and a memory unit • Instructions are executed sequentially and the system may or may not have internal parallel processing capabilities • Parallel processing in this case may be achieved by means of multiple functional units SIMD(Single Instruction Multiple Data)
⮚ Single CU and multiple PEs
⮚ CU fetches an instruction from memory and after decoding, broadcasts control signals to all PEs. ⮚ That is, at any given time, all PEs are Synchronously executing the same. ⮚ Instruction but on different setsof data; hence the name SIMD SIMD(Single Instruction Multiple Data) SIMD(Single Instruction Multiple Data)
⮚ SIMD represent an organization that include many
processing units under the supervision of a common control unit ⮚ All processor receive the same instruction from the control unit but operate on different items of data ⮚ The shared memory unit must contain multiple modules so that it can communicate with all the processors simultaneously Single Instruction, Multiple Data Stream - SIMD • Single machine instruction • Controls simultaneous execution • Number of processing elements • Lockstep basis • Each processing element has associated data memory • Each instruction executed on different set of data by different processors • Vector and array processors MISD(Multiple instructions single data)
⮚ A rare type, since data throughput is limited.
MISD structure is only of theoretical interest since no practical system has been constructed using this organization Multiple Instruction, Single Data Stream - MISD • Sequence of data • Transmitted to set of processors • Each processor executes different instruction sequence • Never been implemented MIMD(Multiple instruction Multiple Data)
⮚ A MIMD is a true multiprocessor
⮚ In contrast to SIMD, a MIMD is a general- purpose machine. ⮚ When all the processor in MIMD are running the same program, we call it Single Program Multiple Data(SPMD) computation. ⮚ The SPMD model is widely used by many parallel platforms. MIMD(Multiple instruction Multiple Data) Multiple Instruction, Multiple Data Stream- MIMD • Set of processors • Simultaneously execute different instruction sequences • Different sets of data • SMPs, clusters and NUMA systems MIMD - Overview • General purpose processors • Each can process all instructions necessary • Further classified by method of processor communication Pipeline Computers • Pipelining is a technique of decomposing a sequential process into sub operations with each sub process being executed in a special dedicated segment that operate concurrently with all other segments • A pipeline can be visualized as a collection of processing segments through which binary information flows • Each segment performs partial processing dictated by the way the task is portioned • The result obtained from the computation in each segment is transferred to the next segment in the pipeline • The final result is obtained after the data have passed throuhg all the segments Pipeline Computers • The name pipeline implies a flow of information analogous to an industrial assembly line • It is characteristics of pipelines that several computations can be in progress in distinct segment at the same time • The overlapping of computation is made possible by associating a register with each segment in the pipeline • The registers provide isolation between each segment so that each can operate on distinct data simultaneously Pipeline Computers • Pipeline structure is imagine that each segment consist of an input register followed by a combinational circuit • The register holds the data and the combinational circuit performs the sub operation in the particular segment • The output of the combinational circuit in a given segment is applied to the input register of the next segment • clock is applied to all registers after enough time has elapsed to perform all segment activity • In this way the information flows through the pipeline one step at a time Pipeline Computers • The pipeline organization example. Suppose we want to perform the combined multiply and add operations with a stream of numbers • Ai * Bi + Ci i=1,2,3,4,5,6,7 • Each sub operation is to be implemented in a segment within a pipeline • Each segment has one or two registers and a combinational circuit shown in the figure • R1 through R5 are registers that receive new data with every clock pulse. The multiplier and adder are combinational circuit • The sub operation performed in each segment of the pipeline are as follows Pipeline Computers •R1<-Ai R2<-Bi input Ai and Bi •R3<-R1*R2 R4<-Ci multiply and input Ci •R5<-R3+R4 Add Ci to product •The five registers are loaded with new data every clock pulse. •The first clock pulse transfers A1 and B1 into R1 and R2. •The second clock pulse transfers the product of r1 and r2 into R3 and C1 into r4 •The same clock pulse transfer A2 and B2 into R1 and r2 •The third clock pulse operates on all three segments simultaneously Pipeline Computers •It pluses A3 and B3 into r1 and r2 transfers the product of r1 and R2 into R3 transfers C2 into r4 and places the sum of R3 and r4 into r5. •It take 3 clock pulses to fill up the pipe and retrieve the first output from r5 •From there on each clock produces a new output and moves the data one step down the pipeline •When no more input data available ,the clock must continue until the last output emerges out of the pipeline Pipeline Computers Pipeline Computers