Lecture4
Download as PPTX, PDF•0 likes•1,084 views
This document discusses high performance computing and Flynn's taxonomy of computer architectures. It describes MIMD architectures including shared memory SMP systems and distributed memory clusters. SMP systems have multiple similar processors that share main memory and I/O. Clusters are groups of interconnected computers that function as a single system. The document compares SMP and cluster architectures.
Lecture4
- 1. High Performance Computing Jawwad Shamsi Lecture #4 25th January 2010
- 2. Recap • Pipelining • Super Scalar Execution – Dependencies • Branch • Data • Resource • Effect of Latency and Memory • Effect of Parallelism
- 3. Flynn’s taxanomy It distinguishes multiprocessor computers according to data and instruction • the dimensions of Instruction and Data • SISD: Single Instruction Single data (Uniprocessor) • SIMD: Single Instruction Multiple data (Vector Processing) • MISD: Multiple Instruction Single date • MIMD: Multiple Instruction Multiple data (SMP, cluster, NUMA)
- 4. MIMD • MIMD – Shared Memory (tightly coupled) • SMP (Symmetric Multiprocessing) • Non-Uniform Memory access – Distributed Memory (loosely coupled) • Clusters
- 5. Taxonomy of Parallel Processor Architectures
- 6. Shared Address Space • Shared Memory • Distributed Memory
- 7. SMP • Two or more similar processors • Same main memory and I/O • Can perform similar operations • Share access to I/O devices
- 8. Multiprogramming and Multiprocessing
- 9. SMP Advantages • Performance • Availability • Incremental growth • Scaling
- 10. Block Diagram of Tightly Coupled Multiprocessor
- 11. Cache Coherence • Multiple copies of cache can maintain different data – Protocols?
- 12. Processor Design: Modes of Parallelism • Two ways to increase parallelism – Superscaling • Instruction level parallelism – Threading • Thread level parallelism – Concept of Multithreaded processors » May or may not be different than OS level mult-threading • Temporal Multi-threading (also called implicit) – Instructions from only one thread • Simultaneous Multi-threading (explicit) – Instructions from more than one thread can be executed
- 13. Scalar Processor Approaches • Single-threaded scalar – Simple pipeline – No multithreading • Interleaved multithreaded scalar – Easiest multithreading to implement – Switch threads at each clock cycle – Pipeline stages kept close to fully occupied – Hardware needs to switch thread context between cycles • Blocked multithreaded scalar – Thread executed until latency event occurs – Would stop pipeline – Processor switches to another thread
- 14. Clusters • Alternative to SMP • High performance • High availability • Server applications • A group of interconnected whole computers • Working together as unified resource • Illusion of being one machine • Each computer called a node
- 15. Cluster Benefits • Absolute scalability • Incremental scalability • High availability • Superior price/performance
- 16. Cluster Configurations - Standby Server, No Shared Disk
- 17. Cluster v. SMP • Both provide multiprocessor support to high demand applications. • Both available commercially – SMP for longer • SMP: – Easier to manage and control – Closer to single processor systems • Scheduling is main difference • Less physical space • Lower power consumption • Clustering: – Superior incremental & absolute scalability – Superior availability • Redundancy