SlideShare a Scribd company logo

Kinetic Dependence Graphs

•
D
Donald Nguyen

The document discusses kinetic dependence graphs, which are directed acyclic graphs (DAGs) used in program scheduling to manage task dependencies during execution. It highlights the importance of safe-source tests, task execution updates, and barriers within the scheduling process, while also addressing limitations of dependence graphs in dynamic task creation and execution. The paper outlines the potential for improvements in scheduling through various program properties and specialized implementations of kinetic dependence graphs.

Software
1 of 69
Download to read offline
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
Kinetic Dependence Graphs M. Amber Hassaan, Donald Nguyen, Keshav Pingali The University of Texas at Austin
Dependence Graph Scheduling 2 Application Programmer A[N,N] = ... for i in 1:N for j in 1:N A[i,j] = A[i-1,j] + A[i,j-1]
Dependence Graph Scheduling 3 Application Programmer Compiler i j 1 1 2 3 2 3 A[N,N] = ... for i in 1:N for j in 1:N A[i,j] = A[i-1,j] + A[i,j-1]
Dependence Graph Scheduling 4 Application Programmer Compiler RuntimeScheduler i j 1 1 2 3 2 3 A[N,N] = ... for i in 1:N for j in 1:N A[i,j] = A[i-1,j] + A[i,j-1]
Dependence Graph Scheduling 5 Application Programmer Compiler RuntimeScheduler i j 1 1 2 3 2 3 A[N,N] = ... for i in 1:N for j in 1:N A[i,j] = A[i-1,j] + A[i,j-1]
Dependence Graph Scheduling 6 Application Programmer Compiler RuntimeScheduler i j 1 1 2 3 2 3 A[N,N] = ... for i in 1:N for j in 1:N A[i,j] = A[i-1,j] + A[i,j-1] Parallel Task Library
Dependence Graph Scheduling 7 Application Programmer Compiler RuntimeScheduler i j 1 1 2 3 2 3 A[N,N] = ... for i in 1:N for j in 1:N A[i,j] = A[i-1,j] + A[i,j-1] Parallel Task Library Parallel Programming Model
Dependence Graph Scheduling 8 Application Programmer Compiler RuntimeScheduler i j 1 1 2 3 2 3 A[N,N] = ... for i in 1:N for j in 1:N A[i,j] = A[i-1,j] + A[i,j-1] Parallel Task Library Parallel Programming Model …
Dependence Graph Scheduling 9 σ0 Application Programmer Compiler RuntimeScheduler i j 1 1 2 3 2 3 A[N,N] = ... for i in 1:N for j in 1:N A[i,j] = A[i-1,j] + A[i,j-1] Parallel Task Library Parallel Programming Model …
Dependence Graph Scheduling 10 G0 σ0 Application Programmer Compiler RuntimeScheduler i j 1 1 2 3 2 3 A[N,N] = ... for i in 1:N for j in 1:N A[i,j] = A[i-1,j] + A[i,j-1] Parallel Task Library Parallel Programming Model …
Dependence Graph Scheduling 11 G0 σ0 w0 G1 Application Programmer Compiler RuntimeScheduler i j 1 1 2 3 2 3 A[N,N] = ... for i in 1:N for j in 1:N A[i,j] = A[i-1,j] + A[i,j-1] Parallel Task Library Parallel Programming Model …
Dependence Graph Scheduling 12 G0 σ0 w0 G1 w1 w2 G2 Execute in Parallel Application Programmer Compiler RuntimeScheduler i j 1 1 2 3 2 3 A[N,N] = ... for i in 1:N for j in 1:N A[i,j] = A[i-1,j] + A[i,j-1] Parallel Task Library Parallel Programming Model …
Dependence Graph Scheduling 13 G0 σ0 w0 G1 w1 w2 wn-1 G2 Gn… Application Programmer Compiler RuntimeScheduler i j 1 1 2 3 2 3 A[N,N] = ... for i in 1:N for j in 1:N A[i,j] = A[i-1,j] + A[i,j-1] Parallel Task Library Parallel Programming Model …
Dependence Graph Scheduling 14 G0 σ0 w0 G1 w1 w2 wn-1 G2 Gn… w0 σ1 Application Programmer Compiler RuntimeScheduler i j 1 1 2 3 2 3 A[N,N] = ... for i in 1:N for j in 1:N A[i,j] = A[i-1,j] + A[i,j-1] Parallel Task Library Parallel Programming Model …
Dependence Graph Scheduling 15 G0 σ0 w0 G1 w1 w2 wn-1 G2 Gn… w0 σ1 w1 w2 wn-1σ2 … σn Application Programmer Compiler RuntimeScheduler i j 1 1 2 3 2 3 A[N,N] = ... for i in 1:N for j in 1:N A[i,j] = A[i-1,j] + A[i,j-1] Parallel Task Library Parallel Programming Model …
Dependence Graph Scheduling 16 G0 σ0 w0 G1 w1 w2 wn-1 G2 Gn… w0 σ1 w1 w2 wn-1σ2 … σn Scheduler World Program World Application Programmer Compiler RuntimeScheduler i j 1 1 2 3 2 3 A[N,N] = ... for i in 1:N for j in 1:N A[i,j] = A[i-1,j] + A[i,j-1] Parallel Task Library Parallel Programming Model …
Limitations of Dependence Graphs • Not always sufficient – Discrete event simulation – Billiard ball simulation – Kruskal’s algorithm for MSTs – Asynchronous Variational Integrators – … • Ordered algorithms – Have tasks with algorithm-specific order in which tasks must appear to execute • Can use speculation – But overheads can be high for ordered algorithms 17
Discrete Event Simulation 18 2 A C B 3 8
Discrete Event Simulation 19 A C B 3 8
Discrete Event Simulation 20 4 A C B 3 8
Discrete Event Simulation 21 4 A C B 3 8 rw-set
Discrete Event Simulation 22 4 A C B 3 8
Discrete Event Simulation 23 4 A C B 3 8
Discrete Event Simulation 24 4 A C B 8 5
Discrete Event Simulation 25 A C D B 3 2 8
Discrete Event Simulation 26 A C D B 3 2 8
Discrete Event Simulation 27 A C D B 3 2 32 8 8
Discrete Event Simulation 28 4 A C D 7 B 2 3 8 8
Discrete Event Simulation 29 4 A C D 7 B 2 3 4 7 round 0 round 18 8 8
Discrete Event Simulation 30 6A C D B 2 3 4 7 round 0 round 18 8 8
Discrete Event Simulation 31 6A C D B 2 3 4 7 6 round 0 round 1 round 2 8 8 8 8
Discrete Event Simulation • Inadequacy of dependence graphs – Tasks are created dynamically – Not all sources are safe to execute – Task executionmay require updating DAG 32 6A C D B 2 3 4 7 6 round 0 round 1 round 2 8 8 8 8
• A dependence graph is: – A DAG G for the program state σ – An update rule U to produce the next G after executing task w • Remove source 33 σ0 w0 w1 w2 wn-1 w0 σ1 w1 w2 wn-1σ2 … σn G0 G1 G2 Gn… Execute sources Dependence Graph Dependence Graph
Kinetic Dependence Graph 34 σ0 w0 w1 w2 wn-1 w0 σ1 w1 w2 wn-1σ2 … σn G0 G1 G2 Gn… Execute sources • A kinetic dependence graph is: – A DAG G for the program state σ – A safe-source test P – An update rule U to produce the next G after executing task w • Remove source, update other tasks’ dependencies, … Dependence Graph
Kinetic Dependence Graph 35 σ0 w0 σ1 w1 w2 wn-1σ2 … σn • A kinetic dependence graph is: – A DAG G for the program state σ – A safe-source test P – An update rule U to produce the next G after executing task w • Remove source, update other tasks’ dependencies, … <σ0, G0> <σ1, G1> <σ2, G2> <σn, Gn> w0 w1 w2 wn-1 … Dependence Graph Kinetic
Kinetic Dependence Graph 36 σ0 w0 σ1 w1 w2 wn-1σ2 … σn • A kinetic dependence graph is: – A DAG G for the program state σ – A safe-source test P – An update rule U to produce the next G after executing task w • Remove source, update other tasks’ dependencies, … <σ0, G0> <σ1, G1> <σ2, G2> <σn, Gn> w0 w1 w2 wn-1 … Dependence Graph Kinetic Execute safe sources
Safe Sources • Application-specific cone of influence 37 time x y
Safe Sources • Application-specific cone of influence 38 time x y
Safe Sources • Application-specific cone of influence 39 • In some algorithms, all sources are safe – Stable source algorithm time x y
General KDG Executor 40 a,1 c,3 d,4 b,2
General KDG Executor 41 1. Construct initial DAG a,1 c,3 d,4 b,2
General KDG Executor 42 1. Construct initial DAG a,1 b,2 c,3 d,4 σ0 ,< > a,1 c,3 d,4 b,2
General KDG Executor 43 1. Construct initial DAG 2. Apply safe source test 3. Execute 4. Updaterw-sets 5. Add new tasks a,1 b,2 c,3 d,4 σ0 ,< > a,1 c,3 d,4 b,2
General KDG Executor 44 a,1 1. Construct initial DAG 2. Apply safe source test 3. Execute 4. Updaterw-sets 5. Add new tasks a,1 b,2 c,3 d,4 σ0 ,< > a,1 c,3 d,4 b,2
General KDG Executor 45 a,1 1. Construct initial DAG 2. Apply safe source test 3. Execute 4. Updaterw-sets 5. Add new tasks a,1 b,2 c,3 d,4 σ0 ,< > a,1 c,3 d,4 b,2
General KDG Executor 46 a,1 c,3 d,4 b,2 a,1 1. Construct initial DAG 2. Apply safe source test 3. Execute 4. Updaterw-sets 5. Add new tasks a,1 b,2 c,3 d,4 σ0 ,< > a,1 c,3 d,4 b,2
General KDG Executor 47 a,1 c,3 d,4 b,2 a,1 1. Construct initial DAG 2. Apply safe source test 3. Execute 4. Updaterw-sets 5. Add new tasks a,1 b,2 c,3 d,4 σ0 ,< > b,2 c,3 d,4 ,< > a,1 a,1 c,3 d,4 b,2
General KDG Executor 48 a,1 c,3 d,4 b,2 a,1 1. Construct initial DAG 2. Apply safe source test 3. Execute 4. Updaterw-sets 5. Add new tasks a,1 b,2 c,3 d,4 σ0 ,< > b,2 c,3 d,4 ,< > a,1 a,1 c,3 d,4 b,2
General KDG Executor 49 a,1 c,3 d,4 b,2 a,1 1. Construct initial DAG 2. Apply safe source test 3. Execute 4. Updaterw-sets 5. Add new tasks a,1 b,2 c,3 d,4 σ0 ,< > b,2 c,3 d,4 ,< > a,1 a,1 c,3 d,4 b,2
General KDG Executor 50 c,3 d,4 b,2 a,1 1. Construct initial DAG 2. Apply safe source test 3. Execute 4. Updaterw-sets 5. Add new tasks a,1 b,2 c,3 d,4 σ0 ,< > b,2 c,3 d,4 ,< > a,1 c,3 d,4 b,2
General KDG Executor 51 c,3 d,4 b,2 a,1 1. Construct initial DAG 2. Apply safe source test 3. Execute 4. Updaterw-sets 5. Add new tasks a,1 b,2 c,3 d,4 σ0 ,< > b,2 c,3 d,4 ,< > a,1 c,3 d,4 b,2
General KDG Executor 52 c,3 d,4 b,2 a,1 1. Construct initial DAG 2. Apply safe source test 3. Execute 4. Updaterw-sets 5. Add new tasks a,1 b,2 c,3 d,4 σ0 ,< > b,2 c,3 d,4 ,< > a,1 c,3 d,4 b,2
General KDG Executor 53 c,3 d,4 b,2 a,1 1. Construct initial DAG 2. Apply safe source test 3. Execute 4. Updaterw-sets 5. Add new tasks a,1 b,2 c,3 d,4 σ0 ,< > b,2 c,3 d,4 ,< > a,1 c,3 d,4 b,2
General KDG Executor 54 c,3 d,4 b,2 e,0 a,1 1. Construct initial DAG 2. Apply safe source test 3. Execute 4. Updaterw-sets 5. Add new tasks a,1 b,2 c,3 d,4 σ0 ,< > b,2 c,3 d,4 e,0 ,< > a,1 c,3 d,4 b,2
General KDG Executor 55 c,3 d,4 b,2 e,0 a,1 1. Construct initial DAG 2. Apply safe source test 3. Execute 4. Updaterw-sets 5. Add new tasks a,1 b,2 c,3 d,4 σ0 ,< > b,2 c,3 d,4 e,0 ,< > a,1 c,3 d,4 b,2
General KDG Executor 56 c,3 d,4 b,2 e,0 a,1 1. Construct initial DAG 2. Apply safe source test 3. Execute 4. Updaterw-sets 5. Add new tasks a,1 b,2 c,3 d,4 σ0 ,< > b,2 c,3 d,4 e,0 σ1 ,< > a,1 a,1 c,3 d,4 b,2
KDG Specializations • Each step needs a barrier • Task’s rw-set potentially updated several times prior to execution 57 Opportunities KDG Runtime
KDG Specializations • Each step needs a barrier • Task’s rw-set potentially updated several times prior to execution 58 Opportunities Program Properties Application ProgrammerKDG Runtime
KDG Specializations • Each step needs a barrier • Task’s rw-set potentially updated several times prior to execution 59 • Sources – In general: safe source test could inspect entire state – Local safe source test – Stable source (sources are safe) • RW-sets – In general: task could change other tasks’ RW-sets – Non-increasing RW-set – Structure-based RW-set • New tasks – In general: task could create new tasks at any priority – Monotonic – No new tasks Opportunities Program Properties Application ProgrammerKDG Runtime
KDG Specializations • Each step needs a barrier • Task’s rw-set potentially updated several times prior to execution 60 • Sources – In general: safe source test could inspect entire state – Local safe source test – Stable source (sources are safe) • RW-sets – In general: task could change other tasks’ RW-sets – Non-increasing RW-set – Structure-based RW-set • New tasks – In general: task could create new tasks at any priority – Monotonic – No new tasks Opportunities Program Properties • Remove unnecessary barriers given program properties • Construct KDG for prefix of tasks • Use different DAG representations Specializations Application ProgrammerKDG Runtime
KDG Specializations • Each step needs a barrier • Task’s rw-set potentially updated several times prior to execution 61 • Sources – In general: safe source test could inspect entire state – Local safe source test – Stable source (sources are safe) • RW-sets – In general: task could change other tasks’ RW-sets – Non-increasing RW-set – Structure-based RW-set • New tasks – In general: task could create new tasks at any priority – Monotonic – No new tasks Opportunities Program Properties • Remove unnecessary barriers given program properties • Construct KDG for prefix of tasks • Use different DAG representations Specializations Application ProgrammerKDG Runtime
KDGs in Action • Application programmer – Writes programs with ordered loops • Loop body must use provided data structure library • Loop properties provided via annotations • Loop body must read all elements before modifying any (cautious) • KDG runtime (in Galois system) – Uses library API to track rw- sets at runtime – Selects executor based on annotations 62 Graph g = ... Set<Event> E = ... @hasStructuredRWSets @monotonic foreach Event e in E orderedby e.t process(e, g) if * E.push(newE)
Evaluation • 7 applications – From billiards simulation (hard-to-parallelize) to tree traversal (well supported by prior work) • 2 types of programs – Other • 3rd-party handwritten, application-specific, OpenMP tasks, Cilk – KDG-Auto • Ordered loops + program property annotations • 40-core, shared-memory machine 63
1 6 11 16 21 26 31 Billiards DES MST AVI SpeedupoverBestSequential Other KDG-Auto 64 n/a < 1
1 6 11 16 21 26 31 36 41 LU BFS Tree SpeedupoverBestSequential Other KDG-Auto 65
1 6 11 16 21 26 31 36 41 LU BFS Tree SpeedupoverBestSequential Other KDG-Auto 66 OpenMP tasks Cilk
Summary • Problem – Dependence graph scheduling is insufficient for many ordered programs • Solution – Develop general KDG executor – Specialize executor according to small number of programproperties 67 “It is a mistaketo try to look too far ahead. The chain of destiny can only be grasped one link at a time.” -Winston Churchill
Kinetic Dependence Graphs M. Amber Hassaan, Donald Nguyen, Keshav Pingali The University of Texas at Austin
69

More Related Content

PDF
(AOA) and (AON) Network construction and critical path calculations (using fo...
Dr. Mahmoud Al-Naimi
 
PPT
Project management
Arun Kandukuri
 
PDF
SAC 2018: "AutoPUT: An Automated Technique for Retrofitting Closed Unit Tests...
Keita Tsukamoto
 
PPT
mehmet tanlak - 3
MEHMET TANLAK
 
PDF
Programing Slicing and Its applications
Ankur Jain
 
PPTX
PERT/CPM
Farhanah Noah
 
PDF
A new approach of program slicing
Dr Sandeep Kumar Poonia
 
PDF
Recurrent Neuronal Network tailored for Weather Radar Nowcasting
Andreas Scheidegger
 
(AOA) and (AON) Network construction and critical path calculations (using fo...
Dr. Mahmoud Al-Naimi
 
Project management
Arun Kandukuri
 
SAC 2018: "AutoPUT: An Automated Technique for Retrofitting Closed Unit Tests...
Keita Tsukamoto
 
mehmet tanlak - 3
MEHMET TANLAK
 
Programing Slicing and Its applications
Ankur Jain
 
PERT/CPM
Farhanah Noah
 
A new approach of program slicing
Dr Sandeep Kumar Poonia
 
Recurrent Neuronal Network tailored for Weather Radar Nowcasting
Andreas Scheidegger
 

What's hot (19)

PPT
Slicing of Object-Oriented Programs
Praveen Penumathsa
 
PPT
Time cost optimisation
Charul Sharma
 
PDF
Sw metrics for regression testing
Jyotsna Sharma
 
PDF
Early Results of OpenMP 4.5 Portability on NVIDIA GPUs & CPUs
Jeff Larkin
 
PDF
PERT AND CPM PROJECT MANAGEMENT
salsa moyara
 
PPSX
(Aoa) and (aon) networks
Dr. Mahmoud Al-Naimi
 
PPTX
Network problem cpm and pert BY MITESH KUMAR
Mitesh Kumar
 
PDF
CPM-Updating
Mr. Ramesh Nayaka
 
PDF
Spm
debarati roy
 
PPT
Chap003
soumya.parhi
 
PDF
Engineering design process 3rd edition haik solutions manual
SummerfieldTBL
 
PDF
PGA: A Priority-aware Genetic Algorithm for Task Scheduling in Heterogeneous ...
University of Surrey, United Kingdom
 
PPTX
Quantitative analysis for business decision- Program Evaluation and Review Te...
Chandra Shekar Immani
 
PPT
Algorithm analysis
Akshay Dagar
 
PDF
Project network math solution
lNazrulIslaml
 
PDF
Interactive Verification of Safety-Critical Systems
Daniela Da Cruz
 
PPTX
Networktechniques
AmitLokhande16
 
PDF
Management Network Lec 3
cairo university
 
PPT
Project management teaching
CHIRANJAN SAHA
 
Slicing of Object-Oriented Programs
Praveen Penumathsa
 
Time cost optimisation
Charul Sharma
 
Sw metrics for regression testing
Jyotsna Sharma
 
Early Results of OpenMP 4.5 Portability on NVIDIA GPUs & CPUs
Jeff Larkin
 
PERT AND CPM PROJECT MANAGEMENT
salsa moyara
 
(Aoa) and (aon) networks
Dr. Mahmoud Al-Naimi
 
Network problem cpm and pert BY MITESH KUMAR
Mitesh Kumar
 
CPM-Updating
Mr. Ramesh Nayaka
 
Spm
debarati roy
 
Chap003
soumya.parhi
 
Engineering design process 3rd edition haik solutions manual
SummerfieldTBL
 
PGA: A Priority-aware Genetic Algorithm for Task Scheduling in Heterogeneous ...
University of Surrey, United Kingdom
 
Quantitative analysis for business decision- Program Evaluation and Review Te...
Chandra Shekar Immani
 
Algorithm analysis
Akshay Dagar
 
Project network math solution
lNazrulIslaml
 
Interactive Verification of Safety-Critical Systems
Daniela Da Cruz
 
Networktechniques
AmitLokhande16
 
Management Network Lec 3
cairo university
 
Project management teaching
CHIRANJAN SAHA
 
Ad

Similar to Kinetic Dependence Graphs (20)

PDF
Chapter Five project management in operations research .pdf
AdaneWuduAbebaw
 
PDF
CS-102 DS-class_01_02 Lectures Data .pdf
ssuser034ce1
 
DOCX
1LocationFixed CostsVariable Costs per unitA=BB=CC=DA$85,000260006.docx
drennanmicah
 
PDF
Lecture - Project, Planning and Control.pdf
lucky141651
 
PPT
Algorithm analysis
sumitbardhan
 
DOCX
Scheduling Problems Complete the following problems covering pro.docx
kenjordan97598
 
PPTX
Section-2 (1).pptx .
aliashraf9689
 
PPT
Introduction of Analysis of Algorithm , asymptotic notations
namrabsit
 
PPT
algorithmAnalysisanddatasteucturesof.ppt
maliozer
 
DOCX
EM Assignment 3.docx
HadiKhan958765
 
PDF
LMmanual.pdf
NarutoUzumaki413489
 
PPTX
SCHEDULING IN PROJECT MANAGEMENT PROJECT SCHEDULE MANAGEMENT
Intan Razali
 
PPT
Algorithm analysis basics - Seven Functions/Big-Oh/Omega/Theta
Priyanka Rana
 
PDF
Sthack 2015 - Jonathan "@JonathanSalwan" Salwan - Dynamic Behavior Analysis U...
StHack
 
PDF
St hack2015 dynamic_behavior_analysis_using_binary_instrumentation_jonathan_s...
Jonathan Salwan
 
PDF
Time-Evolving Graph Processing On Commodity Clusters
Jen Aman
 
PPTX
Pert & cpm project management
Rahul Dubey
 
PPT
data unit notes from department of computer science
sdcmcatmk
 
PPT
daaadafrhdncxfbfbgdngfmfhmhagshh_unit_i.ppt
PRASAD BANOTH
 
DOC
software testing
vamshi batchu
 
Chapter Five project management in operations research .pdf
AdaneWuduAbebaw
 
CS-102 DS-class_01_02 Lectures Data .pdf
ssuser034ce1
 
1LocationFixed CostsVariable Costs per unitA=BB=CC=DA$85,000260006.docx
drennanmicah
 
Lecture - Project, Planning and Control.pdf
lucky141651
 
Algorithm analysis
sumitbardhan
 
Scheduling Problems Complete the following problems covering pro.docx
kenjordan97598
 
Section-2 (1).pptx .
aliashraf9689
 
Introduction of Analysis of Algorithm , asymptotic notations
namrabsit
 
algorithmAnalysisanddatasteucturesof.ppt
maliozer
 
EM Assignment 3.docx
HadiKhan958765
 
LMmanual.pdf
NarutoUzumaki413489
 
SCHEDULING IN PROJECT MANAGEMENT PROJECT SCHEDULE MANAGEMENT
Intan Razali
 
Algorithm analysis basics - Seven Functions/Big-Oh/Omega/Theta
Priyanka Rana
 
Sthack 2015 - Jonathan "@JonathanSalwan" Salwan - Dynamic Behavior Analysis U...
StHack
 
St hack2015 dynamic_behavior_analysis_using_binary_instrumentation_jonathan_s...
Jonathan Salwan
 
Time-Evolving Graph Processing On Commodity Clusters
Jen Aman
 
Pert & cpm project management
Rahul Dubey
 
data unit notes from department of computer science
sdcmcatmk
 
daaadafrhdncxfbfbgdngfmfhmhagshh_unit_i.ppt
PRASAD BANOTH
 
software testing
vamshi batchu
 
Ad

Recently uploaded (20)

PPT
Why Reliable Server Maintenance Service in New York is Crucial for Your Business
Sam Vohra
 
PDF
Wondershare Filmora 14.5.20.12999 Crack Full New Version 2025
gsgssg2211
 
PDF
lesson-2-rules-of-netiquette.pdf.bshhsjdj
jasmenrojas249
 
PPTX
Web Testing.pptx528278vshbuqffqhhqiwnwuq
studylike474
 
PPTX
Visualising Data with Scatterplots in IBM SPSS Statistics.pptx
Version 1 Analytics
 
PPTX
Why Use Open Source Reporting Tools for Business Intelligence.pptx
Varsha Nayak
 
PDF
Salesforce Implementation Services Provider.pdf
VALiNTRY360
 
PPTX
Contractor Management Platform and Software Solution for Compliance
SHEQ Network Limited
 
PPTX
slidesgo-unlocking-the-code-the-dynamic-dance-of-variables-and-constants-2024...
kr2589474
 
PDF
The Role of Automation and AI in EHS Management for Data Centers.pdf
TECH EHS Solution
 
PPTX
Presentation about variables and constant.pptx
kr2589474
 
PPTX
TestNG for Java Testing and Automation testing
ssuser0213cb
 
PPTX
Smart Panchayat Raj e-Governance App.pptx
Rohitnikam33
 
PPTX
The-Dawn-of-AI-Reshaping-Our-World.pptxx
parthbhanushali307
 
PPTX
Presentation about variables and constant.pptx
safalsingh810
 
PPTX
oapresentation.pptx
mehatdhavalrajubhai
 
PDF
IEEE-CS Tech Predictions, SWEBOK and Quantum Software: Towards Q-SWEBOK
Hironori Washizaki
 
PDF
Build Multi-agent using Agent Development Kit
FadyIbrahim23
 
PPT
Activate_Methodology_Summary presentatio
annapureddyn
 
PPTX
PFAS Reporting Requirements 2026 Are You Submission Ready Certivo.pptx
Certivo Inc
 
Why Reliable Server Maintenance Service in New York is Crucial for Your Business
Sam Vohra
 
Wondershare Filmora 14.5.20.12999 Crack Full New Version 2025
gsgssg2211
 
lesson-2-rules-of-netiquette.pdf.bshhsjdj
jasmenrojas249
 
Web Testing.pptx528278vshbuqffqhhqiwnwuq
studylike474
 
Visualising Data with Scatterplots in IBM SPSS Statistics.pptx
Version 1 Analytics
 
Why Use Open Source Reporting Tools for Business Intelligence.pptx
Varsha Nayak
 
Salesforce Implementation Services Provider.pdf
VALiNTRY360
 
Contractor Management Platform and Software Solution for Compliance
SHEQ Network Limited
 
slidesgo-unlocking-the-code-the-dynamic-dance-of-variables-and-constants-2024...
kr2589474
 
The Role of Automation and AI in EHS Management for Data Centers.pdf
TECH EHS Solution
 
Presentation about variables and constant.pptx
kr2589474
 
TestNG for Java Testing and Automation testing
ssuser0213cb
 
Smart Panchayat Raj e-Governance App.pptx
Rohitnikam33
 
The-Dawn-of-AI-Reshaping-Our-World.pptxx
parthbhanushali307
 
Presentation about variables and constant.pptx
safalsingh810
 
oapresentation.pptx
mehatdhavalrajubhai
 
IEEE-CS Tech Predictions, SWEBOK and Quantum Software: Towards Q-SWEBOK
Hironori Washizaki
 
Build Multi-agent using Agent Development Kit
FadyIbrahim23
 
Activate_Methodology_Summary presentatio
annapureddyn
 
PFAS Reporting Requirements 2026 Are You Submission Ready Certivo.pptx
Certivo Inc
 

Kinetic Dependence Graphs

  • 1. Kinetic Dependence Graphs M. Amber Hassaan, Donald Nguyen, Keshav Pingali The University of Texas at Austin
  • 2. Dependence Graph Scheduling 2 Application Programmer A[N,N] = ... for i in 1:N for j in 1:N A[i,j] = A[i-1,j] + A[i,j-1]
  • 3. Dependence Graph Scheduling 3 Application Programmer Compiler i j 1 1 2 3 2 3 A[N,N] = ... for i in 1:N for j in 1:N A[i,j] = A[i-1,j] + A[i,j-1]
  • 4. Dependence Graph Scheduling 4 Application Programmer Compiler RuntimeScheduler i j 1 1 2 3 2 3 A[N,N] = ... for i in 1:N for j in 1:N A[i,j] = A[i-1,j] + A[i,j-1]
  • 5. Dependence Graph Scheduling 5 Application Programmer Compiler RuntimeScheduler i j 1 1 2 3 2 3 A[N,N] = ... for i in 1:N for j in 1:N A[i,j] = A[i-1,j] + A[i,j-1]
  • 6. Dependence Graph Scheduling 6 Application Programmer Compiler RuntimeScheduler i j 1 1 2 3 2 3 A[N,N] = ... for i in 1:N for j in 1:N A[i,j] = A[i-1,j] + A[i,j-1] Parallel Task Library
  • 7. Dependence Graph Scheduling 7 Application Programmer Compiler RuntimeScheduler i j 1 1 2 3 2 3 A[N,N] = ... for i in 1:N for j in 1:N A[i,j] = A[i-1,j] + A[i,j-1] Parallel Task Library Parallel Programming Model
  • 8. Dependence Graph Scheduling 8 Application Programmer Compiler RuntimeScheduler i j 1 1 2 3 2 3 A[N,N] = ... for i in 1:N for j in 1:N A[i,j] = A[i-1,j] + A[i,j-1] Parallel Task Library Parallel Programming Model …
  • 9. Dependence Graph Scheduling 9 σ0 Application Programmer Compiler RuntimeScheduler i j 1 1 2 3 2 3 A[N,N] = ... for i in 1:N for j in 1:N A[i,j] = A[i-1,j] + A[i,j-1] Parallel Task Library Parallel Programming Model …
  • 10. Dependence Graph Scheduling 10 G0 σ0 Application Programmer Compiler RuntimeScheduler i j 1 1 2 3 2 3 A[N,N] = ... for i in 1:N for j in 1:N A[i,j] = A[i-1,j] + A[i,j-1] Parallel Task Library Parallel Programming Model …
  • 11. Dependence Graph Scheduling 11 G0 σ0 w0 G1 Application Programmer Compiler RuntimeScheduler i j 1 1 2 3 2 3 A[N,N] = ... for i in 1:N for j in 1:N A[i,j] = A[i-1,j] + A[i,j-1] Parallel Task Library Parallel Programming Model …
  • 12. Dependence Graph Scheduling 12 G0 σ0 w0 G1 w1 w2 G2 Execute in Parallel Application Programmer Compiler RuntimeScheduler i j 1 1 2 3 2 3 A[N,N] = ... for i in 1:N for j in 1:N A[i,j] = A[i-1,j] + A[i,j-1] Parallel Task Library Parallel Programming Model …
  • 13. Dependence Graph Scheduling 13 G0 σ0 w0 G1 w1 w2 wn-1 G2 Gn… Application Programmer Compiler RuntimeScheduler i j 1 1 2 3 2 3 A[N,N] = ... for i in 1:N for j in 1:N A[i,j] = A[i-1,j] + A[i,j-1] Parallel Task Library Parallel Programming Model …
  • 14. Dependence Graph Scheduling 14 G0 σ0 w0 G1 w1 w2 wn-1 G2 Gn… w0 σ1 Application Programmer Compiler RuntimeScheduler i j 1 1 2 3 2 3 A[N,N] = ... for i in 1:N for j in 1:N A[i,j] = A[i-1,j] + A[i,j-1] Parallel Task Library Parallel Programming Model …
  • 15. Dependence Graph Scheduling 15 G0 σ0 w0 G1 w1 w2 wn-1 G2 Gn… w0 σ1 w1 w2 wn-1σ2 … σn Application Programmer Compiler RuntimeScheduler i j 1 1 2 3 2 3 A[N,N] = ... for i in 1:N for j in 1:N A[i,j] = A[i-1,j] + A[i,j-1] Parallel Task Library Parallel Programming Model …
  • 16. Dependence Graph Scheduling 16 G0 σ0 w0 G1 w1 w2 wn-1 G2 Gn… w0 σ1 w1 w2 wn-1σ2 … σn Scheduler World Program World Application Programmer Compiler RuntimeScheduler i j 1 1 2 3 2 3 A[N,N] = ... for i in 1:N for j in 1:N A[i,j] = A[i-1,j] + A[i,j-1] Parallel Task Library Parallel Programming Model …
  • 17. Limitations of Dependence Graphs • Not always sufficient – Discrete event simulation – Billiard ball simulation – Kruskal’s algorithm for MSTs – Asynchronous Variational Integrators – … • Ordered algorithms – Have tasks with algorithm-specific order in which tasks must appear to execute • Can use speculation – But overheads can be high for ordered algorithms 17
  • 29. Discrete Event Simulation 29 4 A C D 7 B 2 3 4 7 round 0 round 18 8 8
  • 30. Discrete Event Simulation 30 6A C D B 2 3 4 7 round 0 round 18 8 8
  • 31. Discrete Event Simulation 31 6A C D B 2 3 4 7 6 round 0 round 1 round 2 8 8 8 8
  • 32. Discrete Event Simulation • Inadequacy of dependence graphs – Tasks are created dynamically – Not all sources are safe to execute – Task executionmay require updating DAG 32 6A C D B 2 3 4 7 6 round 0 round 1 round 2 8 8 8 8
  • 33. • A dependence graph is: – A DAG G for the program state σ – An update rule U to produce the next G after executing task w • Remove source 33 σ0 w0 w1 w2 wn-1 w0 σ1 w1 w2 wn-1σ2 … σn G0 G1 G2 Gn… Execute sources Dependence Graph Dependence Graph
  • 34. Kinetic Dependence Graph 34 σ0 w0 w1 w2 wn-1 w0 σ1 w1 w2 wn-1σ2 … σn G0 G1 G2 Gn… Execute sources • A kinetic dependence graph is: – A DAG G for the program state σ – A safe-source test P – An update rule U to produce the next G after executing task w • Remove source, update other tasks’ dependencies, … Dependence Graph
  • 35. Kinetic Dependence Graph 35 σ0 w0 σ1 w1 w2 wn-1σ2 … σn • A kinetic dependence graph is: – A DAG G for the program state σ – A safe-source test P – An update rule U to produce the next G after executing task w • Remove source, update other tasks’ dependencies, … <σ0, G0> <σ1, G1> <σ2, G2> <σn, Gn> w0 w1 w2 wn-1 … Dependence Graph Kinetic
  • 36. Kinetic Dependence Graph 36 σ0 w0 σ1 w1 w2 wn-1σ2 … σn • A kinetic dependence graph is: – A DAG G for the program state σ – A safe-source test P – An update rule U to produce the next G after executing task w • Remove source, update other tasks’ dependencies, … <σ0, G0> <σ1, G1> <σ2, G2> <σn, Gn> w0 w1 w2 wn-1 … Dependence Graph Kinetic Execute safe sources
  • 37. Safe Sources • Application-specific cone of influence 37 time x y
  • 38. Safe Sources • Application-specific cone of influence 38 time x y
  • 39. Safe Sources • Application-specific cone of influence 39 • In some algorithms, all sources are safe – Stable source algorithm time x y
  • 41. General KDG Executor 41 1. Construct initial DAG a,1 c,3 d,4 b,2
  • 42. General KDG Executor 42 1. Construct initial DAG a,1 b,2 c,3 d,4 σ0 ,< > a,1 c,3 d,4 b,2
  • 43. General KDG Executor 43 1. Construct initial DAG 2. Apply safe source test 3. Execute 4. Updaterw-sets 5. Add new tasks a,1 b,2 c,3 d,4 σ0 ,< > a,1 c,3 d,4 b,2
  • 44. General KDG Executor 44 a,1 1. Construct initial DAG 2. Apply safe source test 3. Execute 4. Updaterw-sets 5. Add new tasks a,1 b,2 c,3 d,4 σ0 ,< > a,1 c,3 d,4 b,2
  • 45. General KDG Executor 45 a,1 1. Construct initial DAG 2. Apply safe source test 3. Execute 4. Updaterw-sets 5. Add new tasks a,1 b,2 c,3 d,4 σ0 ,< > a,1 c,3 d,4 b,2
  • 46. General KDG Executor 46 a,1 c,3 d,4 b,2 a,1 1. Construct initial DAG 2. Apply safe source test 3. Execute 4. Updaterw-sets 5. Add new tasks a,1 b,2 c,3 d,4 σ0 ,< > a,1 c,3 d,4 b,2
  • 47. General KDG Executor 47 a,1 c,3 d,4 b,2 a,1 1. Construct initial DAG 2. Apply safe source test 3. Execute 4. Updaterw-sets 5. Add new tasks a,1 b,2 c,3 d,4 σ0 ,< > b,2 c,3 d,4 ,< > a,1 a,1 c,3 d,4 b,2
  • 48. General KDG Executor 48 a,1 c,3 d,4 b,2 a,1 1. Construct initial DAG 2. Apply safe source test 3. Execute 4. Updaterw-sets 5. Add new tasks a,1 b,2 c,3 d,4 σ0 ,< > b,2 c,3 d,4 ,< > a,1 a,1 c,3 d,4 b,2
  • 49. General KDG Executor 49 a,1 c,3 d,4 b,2 a,1 1. Construct initial DAG 2. Apply safe source test 3. Execute 4. Updaterw-sets 5. Add new tasks a,1 b,2 c,3 d,4 σ0 ,< > b,2 c,3 d,4 ,< > a,1 a,1 c,3 d,4 b,2
  • 50. General KDG Executor 50 c,3 d,4 b,2 a,1 1. Construct initial DAG 2. Apply safe source test 3. Execute 4. Updaterw-sets 5. Add new tasks a,1 b,2 c,3 d,4 σ0 ,< > b,2 c,3 d,4 ,< > a,1 c,3 d,4 b,2
  • 51. General KDG Executor 51 c,3 d,4 b,2 a,1 1. Construct initial DAG 2. Apply safe source test 3. Execute 4. Updaterw-sets 5. Add new tasks a,1 b,2 c,3 d,4 σ0 ,< > b,2 c,3 d,4 ,< > a,1 c,3 d,4 b,2
  • 52. General KDG Executor 52 c,3 d,4 b,2 a,1 1. Construct initial DAG 2. Apply safe source test 3. Execute 4. Updaterw-sets 5. Add new tasks a,1 b,2 c,3 d,4 σ0 ,< > b,2 c,3 d,4 ,< > a,1 c,3 d,4 b,2
  • 53. General KDG Executor 53 c,3 d,4 b,2 a,1 1. Construct initial DAG 2. Apply safe source test 3. Execute 4. Updaterw-sets 5. Add new tasks a,1 b,2 c,3 d,4 σ0 ,< > b,2 c,3 d,4 ,< > a,1 c,3 d,4 b,2
  • 54. General KDG Executor 54 c,3 d,4 b,2 e,0 a,1 1. Construct initial DAG 2. Apply safe source test 3. Execute 4. Updaterw-sets 5. Add new tasks a,1 b,2 c,3 d,4 σ0 ,< > b,2 c,3 d,4 e,0 ,< > a,1 c,3 d,4 b,2
  • 55. General KDG Executor 55 c,3 d,4 b,2 e,0 a,1 1. Construct initial DAG 2. Apply safe source test 3. Execute 4. Updaterw-sets 5. Add new tasks a,1 b,2 c,3 d,4 σ0 ,< > b,2 c,3 d,4 e,0 ,< > a,1 c,3 d,4 b,2
  • 56. General KDG Executor 56 c,3 d,4 b,2 e,0 a,1 1. Construct initial DAG 2. Apply safe source test 3. Execute 4. Updaterw-sets 5. Add new tasks a,1 b,2 c,3 d,4 σ0 ,< > b,2 c,3 d,4 e,0 σ1 ,< > a,1 a,1 c,3 d,4 b,2
  • 57. KDG Specializations • Each step needs a barrier • Task’s rw-set potentially updated several times prior to execution 57 Opportunities KDG Runtime
  • 58. KDG Specializations • Each step needs a barrier • Task’s rw-set potentially updated several times prior to execution 58 Opportunities Program Properties Application ProgrammerKDG Runtime
  • 59. KDG Specializations • Each step needs a barrier • Task’s rw-set potentially updated several times prior to execution 59 • Sources – In general: safe source test could inspect entire state – Local safe source test – Stable source (sources are safe) • RW-sets – In general: task could change other tasks’ RW-sets – Non-increasing RW-set – Structure-based RW-set • New tasks – In general: task could create new tasks at any priority – Monotonic – No new tasks Opportunities Program Properties Application ProgrammerKDG Runtime
  • 60. KDG Specializations • Each step needs a barrier • Task’s rw-set potentially updated several times prior to execution 60 • Sources – In general: safe source test could inspect entire state – Local safe source test – Stable source (sources are safe) • RW-sets – In general: task could change other tasks’ RW-sets – Non-increasing RW-set – Structure-based RW-set • New tasks – In general: task could create new tasks at any priority – Monotonic – No new tasks Opportunities Program Properties • Remove unnecessary barriers given program properties • Construct KDG for prefix of tasks • Use different DAG representations Specializations Application ProgrammerKDG Runtime
  • 61. KDG Specializations • Each step needs a barrier • Task’s rw-set potentially updated several times prior to execution 61 • Sources – In general: safe source test could inspect entire state – Local safe source test – Stable source (sources are safe) • RW-sets – In general: task could change other tasks’ RW-sets – Non-increasing RW-set – Structure-based RW-set • New tasks – In general: task could create new tasks at any priority – Monotonic – No new tasks Opportunities Program Properties • Remove unnecessary barriers given program properties • Construct KDG for prefix of tasks • Use different DAG representations Specializations Application ProgrammerKDG Runtime
  • 62. KDGs in Action • Application programmer – Writes programs with ordered loops • Loop body must use provided data structure library • Loop properties provided via annotations • Loop body must read all elements before modifying any (cautious) • KDG runtime (in Galois system) – Uses library API to track rw- sets at runtime – Selects executor based on annotations 62 Graph g = ... Set<Event> E = ... @hasStructuredRWSets @monotonic foreach Event e in E orderedby e.t process(e, g) if * E.push(newE)
  • 63. Evaluation • 7 applications – From billiards simulation (hard-to-parallelize) to tree traversal (well supported by prior work) • 2 types of programs – Other • 3rd-party handwritten, application-specific, OpenMP tasks, Cilk – KDG-Auto • Ordered loops + program property annotations • 40-core, shared-memory machine 63
  • 64. 1 6 11 16 21 26 31 Billiards DES MST AVI SpeedupoverBestSequential Other KDG-Auto 64 n/a < 1
  • 67. Summary • Problem – Dependence graph scheduling is insufficient for many ordered programs • Solution – Develop general KDG executor – Specialize executor according to small number of programproperties 67 “It is a mistaketo try to look too far ahead. The chain of destiny can only be grasped one link at a time.” -Winston Churchill
  • 68. Kinetic Dependence Graphs M. Amber Hassaan, Donald Nguyen, Keshav Pingali The University of Texas at Austin
  • 69. 69