Ad
More Related Content
What's hot (20)
Similar to Lecture 2 (20)
Ad
More from Mr SMAK (20)
Ad
Recently uploaded (20)
Lecture 2
- 1. Parallel Computing Concepts and Terminologies
- 2. Terminologies โข Task โ A logically discrete section of computational work. A task is typically a program or program-like set of instructions that is executed by a processor. โข Parallel Task โ A task that can be executed by multiple processors safely (yields correct results) โข Serial Execution โ Execution of a program sequentially, one statement at a time. In the simplest sense, this is what happens on a one processor machine. However, virtually all parallel tasks will have sections of a parallel program that must be executed serially. โข Parallel Execution โ Execution of a program by more than one task, with each task being able to execute the same or different statement at the same moment in time.
- 3. Terminologies โข Shared Memory โ From a strictly hardware point of view, describes a computer architecture where all processors have direct (usually bus based) access to common physical memory. In a programming sense, it describes a model where parallel tasks all have the same "picture" of memory and can directly address and access the same logical memory locations regardless of where the physical memory actually exists. โข Distributed Memory โ In hardware, refers to network based memory access for physical memory that is not common. As a programming model, tasks can only logically "see" local machine memory and must use communications to access memory on other machines where other tasks are executing
- 4. Terminologies โข Communications โ Parallel tasks typically need to exchange data. There are several ways this can be accomplished, such as through a shared memory bus or over a network, however the actual event of data exchange is commonly referred to as communications regardless of the method employed. โข Synchronization โ The coordination of parallel tasks in real time, very often associated with communications. Often implemented by establishing a synchronization point within an application where a task may not proceed further until another task(s) reaches the same or logically equivalent point. Synchronization usually involves waiting by at least one task, and can therefore cause a parallel application's wall clock execution time to increase
- 5. Terminologies โข Granularity โ In parallel computing, granularity is a qualitative measure of the ratio of computation to communication. โข Coarse: relatively large amounts of computational work are done between communication events โข Fine: relatively small amounts of computational work are done between communication events โข Observed Speedup โ Observed speedup of a code which has been parallelized, defined as: wall-clock time of serial execution / wall-clock time of parallel execution โ One of the simplest and most widely used indicators for
- 6. von Neumann Architecture โข For over 40 years, virtually all computers have followed a common machine model known as the von Neumann computer. Named after the Hungarian mathematician John von Neumann. โข A von Neumann computer uses the stored- program concept. The CPU executes a stored program that specifies a sequence of read and write operations on the memory.
- 7. von Neumann Architecture โข Memory is used to store both program and data instructions โข Program instructions are coded data which tell the computer to do something โข Data is simply information to be used by the program โข A central processing unit (CPU) gets instructions and/or data from memory, decodes the instructions and then sequentially performs them
- 8. Flynn's Classical Taxonomy โข There are different ways to classify parallel computers. One of the more widely used classifications, in use since 1966, is called Flynn's Taxonomy. โข Flynn's taxonomy distinguishes multi-processor computer architectures according to how they can be classified along the two independent dimensions of Instruction and Data. Each of these dimensions can have only one of two possible states: Single or Multiple.
- 9. Flynn's Classical Taxonomy โข The matrix below defines the 4 possible classifications according to Flynn. โขS I S D โขS I M D โขSingle Instruction, Single Data โขSingle Instruction, Multiple Data โขM I S D โขM I M D โขMultiple Instruction, Single โขMultiple Instruction, Multiple Data Data
- 10. Single Instruction, Single Data (SISD) โข A serial (non-parallel) computer โข Single instruction: only one instruction stream is being acted on by the CPU during any one clock cycle โข Single data: only one data stream is being used as input during any one clock cycle โข Deterministic execution โข This is the oldest and until recently, the most prevalent form of computer โข Eg: most PCs, single CPU workstations and mainframes
- 11. Single Instruction, Multiple Data (SIMD): โข A type of parallel computer โข Single instruction: All processing units execute the same instruction at any given clock cycle โข Multiple data: Each processing unit can operate on a different data element โข This type of machine typically has an instruction dispatcher, a very high-bandwidth internal network, and a very large array of very small- capacity instruction units.
- 12. Single Instruction, Multiple Data (SIMD):
- 13. Single Instruction, Multiple Data (SIMD): โข Best suited for specialized problems characterized by a high degree of regularity,such as image processing. โข Synchronous (lockstep) and deterministic execution โข Two varieties: Processor Arrays and Vector Pipelines โข Examples (some extinct): o Processor Arrays: Connection Machine CM-2, Maspar MP-1, MP-2 o Vector Pipelines: IBM 9000, Cray C90, Fujitsu VP, NEC SX-2, Hitachi S820
- 14. Multiple Instruction, Single Data (MISD): โข Few actual examples of this class of parallel computer have ever existed โข Some conceivable examples might be: โ multiple frequency filters operating on a single signal stream โ multiple cryptography algorithms attempting to crack a single coded message.
- 15. Multiple Instruction, Multiple Data (MIMD): โข Currently, the most common type of parallel computer โข Multiple Instruction: every processor may be executing a different instruction stream โข Multiple Data: every processor may be working with a different data stream โข Execution can be synchronous or asynchronous, deterministic or non- deterministic โข Examples: most current supercomputers, networked parallel computer "grids" and multi-processor SMP computers - including some types of PCs.
- 16. Multiple Instruction, Multiple Data (MIMD):
- 18. scalar sequential Lookahead Functional I/E overlap parallelism Multiple Func. Units Pipeline Implicit Explicit Vector vector Mem-to-Mem Reg-to-Reg SIMD MIMD Assoc. Processor Multicomputer Multiprocessor Processor Array