![For (I=0; I<500; I++)
a[I] = 0;
1970 1980 200620001990
Performance
GO PARALLEL](https://image.slidesharecdn.com/pdclecture1-180408111654/85/Pdc-lecture1-12-320.jpg)
Ad
More Related Content
What's hot (19)
Similar to Pdc lecture1 (20)
Ad
Recently uploaded (20)
Ad
Pdc lecture1
- 1. Parallel and Distributed Systems Instructor: Xin Yuan Department of Computer Science Florida State University
- 2. Parallel and distributed systems • What is a parallel computer? – A collection of processing elements that communicate and coorperate to solve large problems fast. • What is a distributed system? – A collection of independent computers that appear to its users as a single coherent system. – A parallel computer is implicitly a distributed system.
- 3. What are they? • More than one processing element • Elements communicate and cooperate • Appear as one system • PDS hardware is everywhere. – diablo.cs.fsu.edu (2-CPU SMP) – Multicore workstations/laptops – linprog.cs.fsu.edu (4-node cluster, 2-CPU SMP, 8 cores per node) – Hadoop clusters – IBM Blue Gene/L at DOE/NNSA/LLNL (262992 processors). – Data centers – SETI@home.
- 4. Why parallel and distributed computing? • Moore’s law: the number of transistors double every 18 months.
- 5. Why parallel and distributed computing? • How to make good use of the increasing number of transistors on a chip? – Increase the computation width (70’s and 80’s) • 4bit->8bit->16bit->32bit->64bit->…. – Instruction level parallelism (90’s) • Pipeline, LIW, etc – ILP to the extreme (early 00’s) • Out of order execution, 6-way issues, etc – Sequential program performance keeps increasing. • The clock rate keeps increasing, clock cycles get smaller and smaller.
- 6. Why parallel and distributed computing? • The fundament limit of the speed of a sequential processor. – Power wall (high frequency results in heat) – Latency wall (speed of light does not change)
- 7. Power wall: what will happen if we keep pushing ILP?
- 8. Latency wall • Speed of light = 300000000m/s • One cycle at 4Ghz = 0.00000000025s • The distance that the light can move at one cycle: – 0.00000000025 * 300000000 = 7.5cm Intel chip dimension = 1.47 in x 1.47 in = 3.73cm x 3.73cm
- 9. Why parallel and distributed computing? • Power wall and latency wall indicate that the era of single thread performance improvement through Moore’s law is ending/has ended. • More transistors on a chip are now applied to increase system throughput, the total number of instructions executed, but not latency, the time for a job to finish. – Improving ILP improves both. – We see a multi-core era
- 10. • The marching of multi-core (2004-now) – Mainstream processors: • INTEL Quad-core Xeon (4 cores), • AMD Quad-core Opteron (4 cores), • SUN Niagara (8 cores), • IBM Power6 (2 cores) – Others • Intel Tflops (80 cores) • CISCO CSR-1 (180 cores) (network processors) – Increase the throughput without increasing the clock rate.
- 11. • Why parallel and distributed computing? – Programming multi-core systems is fundamentally different from programming traditional computers. • Parallelism needs to be explicitly expressed in the program. – This is traditionally referred to as parallel computing. • PDC is not really a choice anymore.
- 12. For (I=0; I<500; I++) a[I] = 0; 1970 1980 200620001990 Performance GO PARALLEL
- 13. • In the foreseeable future, parallel systems with multi-core processors are going mainstream. – Lots of hardware, software, and human issues in mainstream parallel/distributed computing • How to make use a large number of processors simultaneously • How to write parallel programs easily and efficiently? • What kind of software supports is needed? • Analyzing and optimizing parallel programs is still a hard problem. • How to debug concurrent program?
- 14. Parallel computing issues • Data sharing – single versus multiple address space. • Process coordination – synchronization using locks, messages, and other means. • Distributed versus centralized memory. • Connectivity – single shared bus versus network with many different topologies. • Fault tolerance/reliability.
- 15. Distributed systems issues • Transparency – Access (data representation), location, migration, relocation, replication, concurrency, failure, persistence • Scalability – Size, geographically • Reliability/fault tolerance
- 16. What we will be doing in this course? • This is the first class in PDC. – Systematically introduce most mainstream parallel computing platforms and programming paradigms. • The coverage is not necessary in depth.