Patterns of parallel programming
Download as PPTX, PDF0 likes2,756 views
This document discusses patterns of parallel programming. It begins by explaining why parallel programming is necessary due to limitations of Moore's Law like power consumption and wire delays. It then covers key terms and measures for parallel programming like work, span, speedup and parallelism. Common patterns are overviewed like pipeline, producer-consumer, and Map-Reduce. It warns of dangers like race conditions, deadlocks and starvation. Finally, it provides references for further reading on parallel programming patterns and approaches.
![Fork-Join
• Additional work may be started only when specific subsets of
the original elements have completed processing
• All elements should be given the chance to run even if one
invocation fails (Ping)
Parallel.Invoke(
() => ComputeMean(),
Fork () => ComputeMedian(),
() => ComputeMode());
Compute Compute Compute static void MyParallelInvoke(params Action[] actions)
Median Mean Mode {
var tasks = new Task[actions.Length];
for (int i = 0; i < actions.Length; i++)
tasks[i] = Task.Factory.StartNew(actions[i]);
Join Task.WaitAll(tasks);
}](https://image.slidesharecdn.com/patternsofparallelprogrammingnetusergroup-130311045912-phpapp01/85/Patterns-of-parallel-programming-14-320.jpg)
Ad
More Related Content
What's hot (20)
Viewers also liked (10)
Ad
Similar to Patterns of parallel programming (20)
Ad
More from Alex Tumanoff (20)
Patterns of parallel programming
- 1. Patterns of Parallel Programming Prepared by Yan Drugalya [email protected] @ydrugalya
- 2. Agenda • Why parallel? • Terms and measures • Building Blocks • Patterns overview – Pipeline and data flow – Producer-Consumer – Map-Reduce – Other
- 3. Why Moore's law is not working anymore • Power consumption • Wire delays • DRAM access latency • Diminishing returns of more instruction-level parallelism
- 4. Power consumption Sun’s Surface 10,000 1,000 Rocket Nozzle Power Density (W/cm2) 100 Nuclear Reactor 10 Pentium® processors Hot Plate 1 8080 ‘70 ‘80 ’90 ’00 ‘10
- 5. Wire delays
- 6. Diminishing returns • 80’s – 10 CPI 1 CPI • 90 – 1 CPI 0.5CPI • 00’s: multicore
- 7. No matter how fast processors get, software consistently finds new ways to eat up the extra speed. Herb Sutter
- 8. Survival To scale performance, put many processing cores on the microprocessor chip New Moore’s law edition is about doubling of cores.
- 9. Terms & Measures • Work = T1 • Span = T∞ • Work Law: Tp>=T1/P • Span Law: Tp>=T∞ • Speedup: Tp/T1 – Linear: θ(P) – Perfect: P • Parallelism: T1/T∞ • Tp<=(T1-T∞)/P + T∞
- 10. Definitions • Concurrent - Several things happenings at the same time • Multithreaded – Multiple execution contexts • Parallel – Multiple simultaneous computations • Asynchronous – Not having to wait
- 11. Dangers • Race Conditions • Starvations • Deadlocks • Livelock • Optimizing compilers • …
- 12. Data parallelism Parallel.ForEach(letters, ch => Capitalize(ch));
- 13. Task parallelism Parallel.Invoke(() => Average(), () => Minimum() …);
- 14. Fork-Join • Additional work may be started only when specific subsets of the original elements have completed processing • All elements should be given the chance to run even if one invocation fails (Ping) Parallel.Invoke( () => ComputeMean(), Fork () => ComputeMedian(), () => ComputeMode()); Compute Compute Compute static void MyParallelInvoke(params Action[] actions) Median Mean Mode { var tasks = new Task[actions.Length]; for (int i = 0; i < actions.Length; i++) tasks[i] = Task.Factory.StartNew(actions[i]); Join Task.WaitAll(tasks); }
- 15. Pipeline pattern Task<int> T1 = Task.Factory.StartNew(() => Task 1 { return result1(); }); Task<double> T2 = T1.ContinueWith((antecedent) => Task 2 { return result2(antecedent.Result); }); Task<double> T3 = T2.ContinueWith((antecedent) => Task 3 { return result3(antecedent.Result); });
- 16. Producer/Consumer Disk/Net Read 1 Read 2 Read 3 BlockingCollection<T> Process Process Process
- 18. Other patterns • Speculative Execution • APM (IAsyncResult, Begin/end pairs) • EAP(Operation/Callback pairs)
- 19. References • Patterns for Parallel Programming: Understanding and Applying Parallel Patterns with the .NET Framework 4 • Pluralsight: – Introduction to Async and Parallel Programming in .NET 4 – Async and Parallel Programming: Application Design • The Free Lunch Is Over: A Fundamental Turn Toward Concurrency in Software • Chapter 27 Multithreaded Algorithms from Introduction to algorithms 3rd edition