SlideShare a Scribd company logo

Behind modern concurrency primitives

Download as PPTX, PDF
B
Bartosz Sypytkowski

The document discusses modern concurrency primitives, focusing on concurrency in user and kernel space, thread management, and the use of asynchronous programming patterns like async/await. It compares different approaches such as thread pools, managed threads, and I/O threads and highlights performance considerations when handling tasks in a multi-threaded environment. Various examples illustrate the implications of blocking versus non-blocking I/O operations and detail advanced concepts like coroutine usage and scheduler types.

Technology
1 of 103
Download to read offline
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
BEHIND MODERN CONCURRENCY PRIMITIVES
INTRODUCTION Bartosz Sypytkowski ▪ @Horusiath ▪ b.sypytkowski@gmail.com ▪ bartoszsypytkowski.com
 Concurrency in user vs. kernel space  Thread pools  Schedulers  Coroutines  State machines AGENDA
WHY DO WE USE THIS?
ASYNC/AWAIT Task.Run(async () => { await using var conn = new SqlConnection(ConnectionString); await conn.OpenAsync(); var user = await conn.QueryFirstAsync<User>( "SELECT * FROM Users WHERE Id = @id", new { id = 100 }); Console.WriteLine($"Received user {user.FirstName} {user.LastName}"); });
INSTEAD OF THIS?
THREAD API new Thread(() => { var conn = new SqlConnection(ConnectionString); conn.Open(); var user = conn.QueryFirst<User>( "SELECT * FROM Users WHERE Id = @id", new { id = 100 }); Console.WriteLine($"Received user {user.FirstName} {user.LastName}"); }).Start();
Thread Pool* Managed Thread Managed Thread OS Thread THREADS TASKS User space Kernel space User space Kernel space Managed Thread OS Thread OS Thread Scheduler OS Thread OS Thread OS Thread Task Task Task Task Task Task Task Task Task Task Task Task Task Task Task Task Task Task
THREAD POOLS
THREAD POOL BASICS CPU 1 Agent Managed Thread CPU 2 Agent Managed Thread CPU 3 Agent Managed Thread CPU 4 Agent Managed Thread Job Queue
HARDWARE ARCHITECTURE 101 CPU 1 L1 cache L1 cache L3 cache CPU 2 L1 cache L1 cache RAM CPU L1 cache: 1ns CPU L2 cache: 4-7ns CPU L2 cache: 100ns
PINNING THREAD TO CPU var threads = Process.GetCurrentProcess().Threads; var cpuCount = Environment.ProcessorCount; Console.WriteLine($"Using {threads.Count} threads on {cpuCount} cores."); // => Using 10 threads on 4 cores. for (int i = 0; i < threads.Count; i++) { var pthread = threads[i]; var cpuMask = (1L << (i % cpuCount)); pthread.IdealProcessor = i & cpuCount; // set preferred thread(s) pthread.ProcessorAffinity = (IntPtr)cpuMask; // set thread affinity mask } NOTE: it’s not always possible (iOS) or respected.
HARDWARE ARCHITECTURE NUMA CPU 1 CPU 2 L1-2cache CPU 3 CPU 4 L1-2cache CPU 5 CPU 6 L1-2cache CPU 7 CPU 8 L1-2cache
THREAD POOL MULTIPLE QUEUES CPU 1 Agent Managed Thread CPU 2 Agent Managed Thread CPU 3 Agent Managed Thread CPU 4 Agent Managed Thread
THREAD POOL WORK STEALING CPU 1 Agent Managed Thread CPU 2 Agent Managed Thread CPU 3 Agent Managed Thread CPU 4 Agent Managed Thread Job Job Job Job Job Job Job Job Job Job
THREAD POOL WORK STEALING CPU 1 Agent Managed Thread CPU 2 Agent Managed Thread CPU 3 Agent Managed Thread CPU 4 Agent Managed Thread Job Job Job Job Job Job ? Job Job Job Empty Queue
THREAD POOL WORK STEALING CPU 1 Agent Managed Thread CPU 2 Agent Managed Thread CPU 3 Agent Managed Thread CPU 4 Agent Managed Thread Job Job Job Job Job Job ? Job Job Job
THREAD POOL WORK STEALING CPU 1 Agent Managed Thread CPU 2 Agent Managed Thread CPU 3 Agent Managed Thread CPU 4 Agent Managed Thread Job Job Job Job Job Job Job JobJob
COMPAR ING VECTOR CLOCKS AFFINITY BASED THREAD POOL CPU 1 Agent Managed Thread CPU 2 Agent Managed Thread CPU 3 Agent Managed Thread CPU 4 Agent Managed Thread S1 S1 S1 S1 S2 S2 S3 S2 S3 S3 S4 S5 S4 S4 S5 Always map entity to the same thread / core
AFFINITY THREAD POOL: PERFORMANCE Source: https://scalac.io/improving-akka-dispatchers/
SCENARIO #1 READING A FILE
WHY IS THIS CODE BAD? static async Task ProcessFiles(FileInfo[] files) { var tasks = new Task[files.Length]; for (int i = 0; i < files.Length; i++) tasks[i] = ProcessFile(files[i]); await Task.WhenAll(tasks); } static async Task ProcessFile(FileInfo file) { using var stream = file.OpenRead(); using var reader = new StreamReader(stream); var text = reader.ReadToEnd(); Process(text); }
WHY IS THIS CODE BAD? static async Task ProcessFiles(FileInfo[] files) { var tasks = new Task[files.Length]; for (int i = 0; i < files.Length; i++) tasks[i] = ProcessFile(files[i]); await Task.WhenAll(tasks); } static async Task ProcessFile(FileInfo file) { using var stream = file.OpenRead(); using var reader = new StreamReader(stream); var text = reader.ReadToEnd(); Process(text); } Blocking the thread belonging to a thread pool shared with other tasks.
I/O 1. Just use async I/O API… WHAT CAN WE DO ABOUT IT?
I/O 1. Just use async I/O API… 2. … but what when it’s not possible? WHAT CAN WE DO ABOUT IT?
I/O 1. Just use async I/O API… 2. … but what when it’s not possible? WHAT CAN WE DO ABOUT IT? internal static class LmdbMethods { [DllImport("lmdb", CallingConvention = CallingConvention.Cdecl)] public static extern int mdb_dbi_open(IntPtr txn, string name, DatabaseOpenFlags flags, out uint db); [DllImport("lmdb", CallingConvention = CallingConvention.Cdecl)] public static extern int mdb_cursor_get(IntPtr cursor, ref ValueStructure key, ref ValueStructure data, CursorOperation op); [DllImport("lmdb", CallingConvention = CallingConvention.Cdecl)] public static extern int mdb_cursor_put(IntPtr cursor, ref ValueStructure key, ref ValueStructure value, CursorPutOptions flags); // other methods }
THREAD POOL I/O THREADS CPU 1 Agent Managed Thread CPU 2 Agent Managed Thread CPU 3 Agent Managed Thread CPU 4 Agent Managed Thread Managed Thread I/O Threads Managed Thread Managed Thread
THREAD POOL I/O THREADS CPU 1 Agent Managed Thread CPU 2 Agent Managed Thread CPU 3 Agent Managed Thread CPU 4 Agent Managed Thread Managed Thread I/O Threads Managed Thread Managed Thread J1
THREAD POOL I/O THREADS CPU 1 Agent Managed Thread CPU 2 Agent Managed Thread CPU 3 Agent Managed Thread CPU 4 Agent Managed Thread Managed Thread I/O Threads Managed Thread Managed Thread J1
THREAD POOL I/O THREADS CPU 1 Agent Managed Thread CPU 2 Agent Managed Thread CPU 3 Agent Managed Thread CPU 4 Agent Managed Thread Managed Thread I/O Threads Managed Thread Managed Thread J1
THREAD POOL I/O THREADS CPU 1 Agent Managed Thread CPU 2 Agent Managed Thread CPU 3 Agent Managed Thread CPU 4 Agent Managed Thread Managed Thread I/O Threads Managed Thread Managed Thread J2
GOROUTINES & I/O DEALING WITH KERNEL CALLS
COMPAR ING VECTOR CLOCKS GOROUTINES I/O CALLS CPU 1 Agent Runtime Thread CPU 2 Agent Runtime Thread CPU 3 Agent Runtime Thread CPU 4 Agent Runtime Thread
COMPAR ING VECTOR CLOCKS GOROUTINES I/O CALLS CPU 1 Agent Runtime Thread CPU 2 Agent Runtime Thread CPU 3 Agent Runtime Thread CPU 4 Agent Runtime Thread sys call
COMPAR ING VECTOR CLOCKS GOROUTINES I/O CALLS CPU 1 Agent CPU 2 Agent Runtime Thread CPU 3 Agent Runtime Thread CPU 4 Agent Runtime Thread Runtime Thread sys call
COMPAR ING VECTOR CLOCKS GOROUTINES I/O CALLS CPU 1 Agent CPU 2 Agent Runtime Thread CPU 3 Agent Runtime Thread CPU 4 Agent Runtime Thread Runtime Thread sys call Runtime Thread
SCHEDULERS Preemptive vs. Cooperative
Scheduler depends on coroutines to return control to scheduler. PREEMPTIVE Scheduler is in charge of execution. Coroutines can be preempted. COOPERATIVE
SCENARIO #2 CREATE AN EXCEL FILE
COMPAR ING VECTOR CLOCKS BUILDING AN EXCEL FILE app.get('/valuations.xlsx', async (req, res) => { var workbook = new Excel.Workbook(); var worksheet = workbook.addWorksheet("Valuations"); var valuations = await getValuations(req.params.id); for (let valuation in valuations) { worksheet.addRow(valuation); } await workbook.xlsx.write(res); });
COMPAR ING VECTOR CLOCKS BUILDING AN EXCEL FILE app.get('/valuations.xlsx', async (req, res) => { var workbook = new Excel.Workbook(); var worksheet = workbook.addWorksheet("Valuations"); var valuations = await getValuations(req.params.id); for (let valuation in valuations) { worksheet.addRow(valuation); // 500μs } await workbook.xlsx.write(res); });
COMPAR ING VECTOR CLOCKS BUILDING AN EXCEL FILE app.get('/valuations.xlsx', async (req, res) => { var workbook = new Excel.Workbook(); var worksheet = workbook.addWorksheet("Valuations"); var valuations = await getValuations(req.params.id); for (let valuation in valuations) { // 20,000 items worksheet.addRow(valuation); // 500μs } await workbook.xlsx.write(res); });
COMPAR ING VECTOR CLOCKS BUILDING AN EXCEL FILE app.get('/valuations.xlsx', async (req, res) => { var workbook = new Excel.Workbook(); var worksheet = workbook.addWorksheet("Valuations"); var valuations = await getValuations(req.params.id); for (let valuation in valuations) { // 20,000 items worksheet.addRow(valuation); // 500μs } await workbook.xlsx.write(res); }); Total loop execution time: 10 seconds
HOW TO DEAL WITH LONG RUNNING TASKS?
LONG RUNNING TASK SEPARATE THREAD CPU 1 Agent Managed Thread CPU 2 Agent Managed Thread CPU 3 Agent Managed Thread CPU 4 Agent Managed Thread Scheduler
LONG RUNNING TASK SEPARATE THREAD CPU 1 Agent Managed Thread CPU 2 Agent Managed Thread CPU 3 Agent Managed Thread CPU 4 Agent Managed Thread Scheduler Task Task.Factory.StartNew(DownloadExcel, TaskCreationOptions.LongRunning)
LONG RUNNING TASK SEPARATE THREAD CPU 1 Agent Managed Thread CPU 2 Agent Managed Thread CPU 3 Agent Managed Thread CPU 4 Agent Managed Thread Scheduler Task Task.Factory.StartNew(DownloadExcel, TaskCreationOptions.LongRunning) Managed Thread
LONG RUNNING TASK SEPARATE THREAD CPU 1 Agent Managed Thread CPU 2 Agent Managed Thread CPU 3 Agent Managed Thread CPU 4 Agent Managed Thread Scheduler Managed Thread Task
LONG RUNNING TASK SEPARATE THREAD CPU 1 Agent Managed Thread CPU 2 Agent Managed Thread CPU 3 Agent Managed Thread CPU 4 Agent Managed Thread Scheduler Managed Thread Task
COMPAR ING VECTOR CLOCKS INTRODUCING INTERRUPTION POINTS function park() { return new Promise((resolve) => setTimeout(resolve, 0)); } app.get('/valuations.xlsx', async (req, res) => { var workbook = new Excel.Workbook(); var worksheet = workbook.addWorksheet("Valuations"); var valuations = await getValuations(req.params.id); var i = 0; for (let valuation in valuations) { // 20,000 items if ((i++) % 100 === 0) { await park(); } worksheet.addRow(valuation); // 500μs } await workbook.xlsx.write(res); });
PREEMPTIVE SCHEDULER step-based / time-based
STEP-BASED PREEMPTION % hello world program -module(helloworld). -export([start/0, write/1]). write([]) -> ok; write([File|T]) -> io:fwrite("Writing a file ~p~n", File), write(T). start() -> Files = ["A", "B", "C"], write(Files).
STEP-BASED PREEMPTION Reduction counter under the hood % hello world program -module(helloworld). -export([start/0, write/1]). write([], Reductions) -> ok; write([File|T], Reductions) -> % if Reductions == 0 then yield() io:fwrite("Writing a file ~p~n", File), write(T, Reductions-1). start() -> Files = ["A", "B", "C"], write(Files, 2000).
• JavaScript promises • .NET Task Parallel Library • Java/Scala Futures PREEMPTIVE • OS threads • Go goroutines • Erlang processes COOPERATIVE
COROUTINES eager vs. lazy
let run () = async { let sw = Stopwatch() sw.Start() let t1 = Async.Sleep(5000) let t2 = Async.Sleep(5000) do! t1 do! t2 printfn "Time passed: %ims" sw.ElapsedMilliseconds } static async Task Run() { var sw = new Stopwatch(); sw.Start(); var t1 = Task.Delay(TimeSpan.FromSeconds(5)); var t2 = Task.Delay(TimeSpan.FromSeconds(5)); await t1; await t2; Console.WriteLine( $"Time passed: {sw.ElapsedMilliseconds}ms"); }
let run () = async { let sw = Stopwatch() sw.Start() let t1 = Async.Sleep(5000) let t2 = Async.Sleep(5000) do! t1 do! t2 printfn "Time passed: %ims" sw.ElapsedMilliseconds } static async Task Run() { var sw = new Stopwatch(); sw.Start(); var t1 = Task.Delay(TimeSpan.FromSeconds(5)); var t2 = Task.Delay(TimeSpan.FromSeconds(5)); await t1; await t2; Console.WriteLine( $"Time passed: {sw.ElapsedMilliseconds}ms"); } EAGER LAZY
COROUTINES stackless vs. stackful
• JavaScript promises • .NET Task Parallel Library • Java/Scala Futures STACKFUL • Go goroutines • LUA coroutines • (soon) Java Loom project STACKLESS
COMPAR ING VECTOR CLOCKS STACKLESS COROUTINES OLD CALLBACK API app.get('/valuations.xlsx', function (req, res, next) { var workbook = new Excel.Workbook(); var worksheet = workbook.addWorksheet("Valuations"); getValuations(req.params.id, function (err, valuations) { for (let valuation in valuations) { worksheet.addRow(valuation); } workbook.xlsx.write(res, function (err) { next(); }); }); });
COMPAR ING VECTOR CLOCKS STACKLESS COROUTINES app.get('/valuations.xlsx', (req, res) => { var workbook = new Excel.Workbook(); var worksheet = workbook.addWorksheet("Valuations"); return getValuations(req.params.id) .then((valuations) => { for (let valuation in valuations) { worksheet.addRow(valuation); } return workbook.xlsx.write(res); }); }); OLD PROMISE API
COMPAR ING VECTOR CLOCKS STACKLESS COROUTINES app.get('/valuations.xlsx', async (req, res) => { var workbook = new Excel.Workbook(); var worksheet = workbook.addWorksheet("Valuations"); var valuations = await getValuations(req.params.id); for (let valuation in valuations) { worksheet.addRow(valuation); } await workbook.xlsx.write(res); });
PROBLEM OF FUNCTION COLOURING
STACKFUL COROUTINES STACK main activation frame function foo(a, b) print("foo", a) bar("foo", b) end function bar(caller, x) print(“bar: ", caller) coroutine.yield() print(“bar", x) end co = coroutine.create(foo) coroutine.resume(co, 1, 2) print(“main”) coroutine.resume(co)
STACKFUL COROUTINES STACK main activation frame function foo(a, b) print("foo", a) bar("foo", b) end function bar(caller, x) print(“bar: ", caller) coroutine.yield() print(“bar", x) end co = coroutine.create(foo) coroutine.resume(co, 1, 2) print(“main”) coroutine.resume(co) foo activation frame Start of coroutine stack
STACKFUL COROUTINES STACK main activation frame function foo(a, b) print("foo", a) bar("foo", b) end function bar(caller, x) print(“bar: ", caller) coroutine.yield() print(“bar", x) end co = coroutine.create(foo) coroutine.resume(co, 1, 2) print(“main”) coroutine.resume(co) foo activation frame print activation frame
STACKFUL COROUTINES STACK main activation frame function foo(a, b) print("foo", a) bar("foo", b) end function bar(caller, x) print(“bar: ", caller) coroutine.yield() print(“bar", x) end co = coroutine.create(foo) coroutine.resume(co, 1, 2) print(“main”) coroutine.resume(co) foo activation frame bar activation frame
STACKFUL COROUTINES STACK main activation frame function foo(a, b) print("foo", a) bar("foo", b) end function bar(caller, x) print(“bar: ", caller) coroutine.yield() print(“bar", x) end co = coroutine.create(foo) coroutine.resume(co, 1, 2) print(“main”) coroutine.resume(co) foo activation frame bar activation frame print activation frame
STACKFUL COROUTINES STACK main activation frame function foo(a, b) print("foo", a) bar("foo", b) end function bar(caller, x) print(“bar: ", caller) coroutine.yield() print(“bar", x) end co = coroutine.create(foo) coroutine.resume(co, 1, 2) print(“main”) coroutine.resume(co) foo activation frame bar activation frame Park coroutine stack
STACKFUL COROUTINES STACK main activation frame function foo(a, b) print("foo", a) bar("foo", b) end function bar(caller, x) print(“bar: ", caller) coroutine.yield() print(“bar", x) end co = coroutine.create(foo) coroutine.resume(co, 1, 2) print(“main”) coroutine.resume(co) foo activation frame bar activation frame Park coroutine stack
STACKFUL COROUTINES STACK main activation frame function foo(a, b) print("foo", a) bar("foo", b) end function bar(caller, x) print(“bar: ", caller) coroutine.yield() print(“bar", x) end co = coroutine.create(foo) coroutine.resume(co, 1, 2) print(“main”) coroutine.resume(co) foo activation frame bar activation frame
STACKFUL COROUTINES STACK main activation frame function foo(a, b) print("foo", a) bar("foo", b) end function bar(caller, x) print(“bar: ", caller) coroutine.yield() print(“bar", x) end co = coroutine.create(foo) coroutine.resume(co, 1, 2) print(“main”) coroutine.resume(co) foo activation frame bar activation frame print activation frame
STACKFUL COROUTINES STACK main activation frame function foo(a, b) print("foo", a) bar("foo", b) end function bar(caller, x) print(“bar: ", caller) coroutine.yield() print(“bar", x) end co = coroutine.create(foo) coroutine.resume(co, 1, 2) print(“main”) coroutine.resume(co) foo activation frame bar activation frame Resume coroutine stack
STACKFUL COROUTINES STACK main activation frame function foo(a, b) print("foo", a) bar("foo", b) end function bar(caller, x) print(“bar: ", caller) coroutine.yield() print(“bar", x) end co = coroutine.create(foo) coroutine.resume(co, 1, 2) print(“main”) coroutine.resume(co) foo activation frame bar activation frame
STACKFUL COROUTINES STACK main activation frame function foo(a, b) print("foo", a) bar("foo", b) end function bar(caller, x) print(“bar: ", caller) coroutine.yield() print(“bar", x) end co = coroutine.create(foo) coroutine.resume(co, 1, 2) print(“main”) coroutine.resume(co) foo activation frame bar activation frame print activation frame
WHAT IS THE CORRECT STACK SIZE FOR EACH COROUTINE?
STACKLESS COROUTINES AND YIELD GENERATORS
ASYNC/AWAIT UNITY COROUTINES IEnumerable WaitAndPrint() { print("Starting at " + Time.time); yield return new WaitForSeconds(0.1f); print("Ending at " + Time.time); } async Task WaitAndPrint() { Console.WriteLine("Starting at " + DateTime.Now); await Task.Delay(100); Console.WriteLine("Ending at " + DateTime.Now); }
DEMO Compose Tasks with LINQ
BEHIND ASYNC/AWAIT
WHAT YOU SEE WHAT YOU DON’T SEE static class Program { static async Task Foo(int a, int b) { Console.WriteLine("Foo", a); await Task.Yield(); Console.WriteLine("Foo", b); } } struct Foo__0 : IStateMachine { AsyncTaskMethodBuilder builder; int state; int a; int b; public void MoveNext() { switch (this.state) { case -1: Console.WriteLine("Foo", this.a); var awaiter = Taks.Yield().GetAwaiter(); this.state = 0; this.builder.AwaitUnsafeOnCompleted(ref awaiter, ref this); // ^ awaiter.OnComplete(() => // ThreadPool.QueueUserWorkItem(this.MoveNext)); return; case 0: Console.WriteLine("Foo", this.b); builder.SetResult(); return; } } } static class Program { static Task Foo(int a, int b) { var builder = AsyncTaskMethodBuilder.Create(); var stateMachine = new Foo__0(); // initialize state machine fields stateMachine.a = a; stateMachine.b = b; stateMachine.state = -1; stateMachine.builder = builder; builder.Start(ref stateMachine); // ^ similar to: ThreadPool.QueueUserWorkItem(this.MoveNext); return builder.Task; } }
WHAT YOU SEE WHAT YOU DON’T SEE static class Program { static async Task Foo(int a, int b) { Console.WriteLine("Foo", a); await Task.Yield(); Console.WriteLine("Foo", b); } } struct Foo__0 : IStateMachine { AsyncTaskMethodBuilder builder; int state; int a; int b; public void MoveNext() { switch (this.state) { case -1: Console.WriteLine("Foo", this.a); var awaiter = Taks.Yield().GetAwaiter(); this.state = 0; this.builder.AwaitUnsafeOnCompleted(ref awaiter, ref this); // ^ awaiter.OnComplete(() => // ThreadPool.QueueUserWorkItem(this.MoveNext)); return; case 0: Console.WriteLine("Foo", this.b); builder.SetResult(); return; } } } static class Program { static Task Foo(int a, int b) { var builder = AsyncTaskMethodBuilder.Create(); var stateMachine = new Foo__0(); // initialize state machine fields stateMachine.a = a; stateMachine.b = b; stateMachine.state = -1; stateMachine.builder = builder; builder.Start(ref stateMachine); // ^ similar to: ThreadPool.QueueUserWorkItem(this.MoveNext); return builder.Task; } }
WHAT YOU SEE WHAT YOU DON’T SEE static class Program { static async Task Foo(int a, int b) { Console.WriteLine("Foo", a); await Task.Yield(); Console.WriteLine("Foo", b); } } struct Foo__0 : IStateMachine { AsyncTaskMethodBuilder builder; int state; int a; int b; public void MoveNext() { switch (this.state) { case -1: Console.WriteLine("Foo", this.a); var awaiter = Taks.Yield().GetAwaiter(); this.state = 0; this.builder.AwaitUnsafeOnCompleted(ref awaiter, ref this); // ^ awaiter.OnComplete(() => // ThreadPool.QueueUserWorkItem(this.MoveNext)); return; case 0: Console.WriteLine("Foo", this.b); builder.SetResult(); return; } } } static class Program { static Task Foo(int a, int b) { var builder = AsyncTaskMethodBuilder.Create(); var stateMachine = new Foo__0(); // initialize state machine fields stateMachine.a = a; stateMachine.b = b; stateMachine.state = -1; stateMachine.builder = builder; builder.Start(ref stateMachine); // ^ similar to: ThreadPool.QueueUserWorkItem(this.MoveNext); return builder.Task; } }
WHAT YOU SEE WHAT YOU DON’T SEE static class Program { static async Task Foo(int a, int b) { Console.WriteLine("Foo", a); await Task.Yield(); Console.WriteLine("Foo", b); } } struct Foo__0 : IStateMachine { AsyncTaskMethodBuilder builder; int state; int a; int b; public void MoveNext() { switch (this.state) { case -1: Console.WriteLine("Foo", this.a); var awaiter = Taks.Yield().GetAwaiter(); this.state = 0; this.builder.AwaitUnsafeOnCompleted(ref awaiter, ref this); // ^ awaiter.OnComplete(() => // ThreadPool.QueueUserWorkItem(this.MoveNext)); return; case 0: Console.WriteLine("Foo", this.b); builder.SetResult(); return; } } } static class Program { static Task Foo(int a, int b) { var builder = AsyncTaskMethodBuilder.Create(); var stateMachine = new Foo__0(); // initialize state machine fields stateMachine.a = a; stateMachine.b = b; stateMachine.state = -1; stateMachine.builder = builder; builder.Start(ref stateMachine); // ^ similar to: ThreadPool.QueueUserWorkItem(this.MoveNext); return builder.Task; } }
WHAT YOU SEE WHAT YOU DON’T SEE static class Program { static async Task Foo(int a, int b) { Console.WriteLine("Foo", a); await Task.Yield(); Console.WriteLine("Foo", b); } } struct Foo__0 : IStateMachine { AsyncTaskMethodBuilder builder; int state; int a; int b; public void MoveNext() { switch (this.state) { case -1: Console.WriteLine("Foo", this.a); var awaiter = Taks.Yield().GetAwaiter(); this.state = 0; this.builder.AwaitUnsafeOnCompleted(ref awaiter, ref this); // ^ awaiter.OnComplete(() => // ThreadPool.QueueUserWorkItem(this.MoveNext)); return; case 0: Console.WriteLine("Foo", this.b); builder.SetResult(); return; } } } static class Program { static Task Foo(int a, int b) { var builder = AsyncTaskMethodBuilder.Create(); var stateMachine = new Foo__0(); // initialize state machine fields stateMachine.a = a; stateMachine.b = b; stateMachine.state = -1; stateMachine.builder = builder; builder.Start(ref stateMachine); // ^ similar to: ThreadPool.QueueUserWorkItem(this.MoveNext); return builder.Task; } }
WHAT YOU SEE WHAT YOU DON’T SEE static class Program { static async Task Foo(int a, int b) { Console.WriteLine("Foo", a); await Task.Yield(); Console.WriteLine("Foo", b); } } struct Foo__0 : IStateMachine { AsyncTaskMethodBuilder builder; int state; int a; int b; public void MoveNext() { switch (this.state) { case -1: Console.WriteLine("Foo", this.a); var awaiter = Taks.Yield().GetAwaiter(); this.state = 0; this.builder.AwaitUnsafeOnCompleted(ref awaiter, ref this); // ^ awaiter.OnComplete(() => // ThreadPool.QueueUserWorkItem(this.MoveNext)); return; case 0: Console.WriteLine("Foo", this.b); builder.SetResult(); return; } } } static class Program { static Task Foo(int a, int b) { var builder = AsyncTaskMethodBuilder.Create(); var stateMachine = new Foo__0(); // initialize state machine fields stateMachine.a = a; stateMachine.b = b; stateMachine.state = -1; stateMachine.builder = builder; builder.Start(ref stateMachine); // ^ similar to: ThreadPool.QueueUserWorkItem(this.MoveNext); return builder.Task; } }
WHAT YOU SEE WHAT YOU DON’T SEE static class Program { static async Task Foo(int a, int b) { Console.WriteLine("Foo", a); await Task.Yield(); Console.WriteLine("Foo", b); } } struct Foo__0 : IStateMachine { AsyncTaskMethodBuilder builder; int state; int a; int b; public void MoveNext() { switch (this.state) { case -1: Console.WriteLine("Foo", this.a); var awaiter = Taks.Yield().GetAwaiter(); this.state = 0; this.builder.AwaitUnsafeOnCompleted(ref awaiter, ref this); // ^ awaiter.OnComplete(() => // ThreadPool.QueueUserWorkItem(this.MoveNext)); return; case 0: Console.WriteLine("Foo", this.b); builder.SetResult(); return; } } } static class Program { static Task Foo(int a, int b) { var builder = AsyncTaskMethodBuilder.Create(); var stateMachine = new Foo__0(); // initialize state machine fields stateMachine.a = a; stateMachine.b = b; stateMachine.state = -1; stateMachine.builder = builder; builder.Start(ref stateMachine); // ^ similar to: ThreadPool.QueueUserWorkItem(this.MoveNext); return builder.Task; } }
WHAT YOU SEE WHAT YOU DON’T SEE static class Program { static async Task Foo(int a, int b) { Console.WriteLine("Foo", a); await Task.Yield(); Console.WriteLine("Foo", b); } } struct Foo__0 : IStateMachine { AsyncTaskMethodBuilder builder; int state; int a; int b; public void MoveNext() { switch (this.state) { case -1: Console.WriteLine("Foo", this.a); var awaiter = Taks.Yield().GetAwaiter(); this.state = 0; this.builder.AwaitUnsafeOnCompleted(ref awaiter, ref this); // ^ awaiter.OnComplete(() => // ThreadPool.QueueUserWorkItem(this.MoveNext)); return; case 0: Console.WriteLine("Foo", this.b); builder.SetResult(); return; } } } static class Program { static Task Foo(int a, int b) { var builder = AsyncTaskMethodBuilder.Create(); var stateMachine = new Foo__0(); // initialize state machine fields stateMachine.a = a; stateMachine.b = b; stateMachine.state = -1; stateMachine.builder = builder; builder.Start(ref stateMachine); // ^ similar to: ThreadPool.QueueUserWorkItem(this.MoveNext); return builder.Task; } }
WHAT YOU SEE WHAT YOU DON’T SEE static class Program { static async Task Foo(int a, int b) { Console.WriteLine("Foo", a); await Task.Yield(); Console.WriteLine("Foo", b); } } struct Foo__0 : IStateMachine { AsyncTaskMethodBuilder builder; int state; int a; int b; public void MoveNext() { switch (this.state) { case -1: Console.WriteLine("Foo", this.a); var awaiter = Taks.Yield().GetAwaiter(); this.state = 0; this.builder.AwaitUnsafeOnCompleted(ref awaiter, ref this); // ^ awaiter.OnComplete(() => // ThreadPool.QueueUserWorkItem(this.MoveNext)); return; case 0: Console.WriteLine("Foo", this.b); builder.SetResult(); return; } } } static class Program { static Task Foo(int a, int b) { var builder = AsyncTaskMethodBuilder.Create(); var stateMachine = new Foo__0(); // initialize state machine fields stateMachine.a = a; stateMachine.b = b; stateMachine.state = -1; stateMachine.builder = builder; builder.Start(ref stateMachine); // ^ similar to: ThreadPool.QueueUserWorkItem(this.MoveNext); return builder.Task; } }
WHAT YOU SEE WHAT YOU DON’T SEE static class Program { static async Task Foo(int a, int b) { Console.WriteLine("Foo", a); await Task.Yield(); Console.WriteLine("Foo", b); } } struct Foo__0 : IStateMachine { AsyncTaskMethodBuilder builder; int state; int a; int b; public void MoveNext() { switch (this.state) { case -1: Console.WriteLine("Foo", this.a); var awaiter = Taks.Yield().GetAwaiter(); this.state = 0; this.builder.AwaitUnsafeOnCompleted(ref awaiter, ref this); // ^ awaiter.OnComplete(() => // ThreadPool.QueueUserWorkItem(this.MoveNext)); return; case 0: Console.WriteLine("Foo", this.b); builder.SetResult(); return; } } } static class Program { static Task Foo(int a, int b) { var builder = AsyncTaskMethodBuilder.Create(); var stateMachine = new Foo__0(); // initialize state machine fields stateMachine.a = a; stateMachine.b = b; stateMachine.state = -1; stateMachine.builder = builder; builder.Start(ref stateMachine); // ^ similar to: ThreadPool.QueueUserWorkItem(this.MoveNext); return builder.Task; } }
WHAT YOU SEE WHAT YOU DON’T SEE static class Program { static async Task Foo(int a, int b) { Console.WriteLine("Foo", a); await Task.Yield(); Console.WriteLine("Foo", b); } } struct Foo__0 : IStateMachine { AsyncTaskMethodBuilder builder; int state; int a; int b; public void MoveNext() { switch (this.state) { case -1: Console.WriteLine("Foo", this.a); var awaiter = Taks.Yield().GetAwaiter(); this.state = 0; this.builder.AwaitUnsafeOnCompleted(ref awaiter, ref this); // ^ awaiter.OnComplete(() => // ThreadPool.QueueUserWorkItem(this.MoveNext)); return; case 0: Console.WriteLine("Foo", this.b); builder.SetResult(); return; } } } static class Program { static Task Foo(int a, int b) { var builder = AsyncTaskMethodBuilder.Create(); var stateMachine = new Foo__0(); // initialize state machine fields stateMachine.a = a; stateMachine.b = b; stateMachine.state = -1; stateMachine.builder = builder; builder.Start(ref stateMachine); // ^ similar to: ThreadPool.QueueUserWorkItem(this.MoveNext); return builder.Task; } }
WHAT YOU SEE WHAT YOU DON’T SEE static class Program { static async Task Foo(int a, int b) { Console.WriteLine("Foo", a); await Task.Yield(); Console.WriteLine("Foo", b); } } struct Foo__0 : IStateMachine { AsyncTaskMethodBuilder builder; int state; int a; int b; public void MoveNext() { switch (this.state) { case -1: Console.WriteLine("Foo", this.a); var awaiter = Taks.Yield().GetAwaiter(); this.state = 0; this.builder.AwaitUnsafeOnCompleted(ref awaiter, ref this); // ^ awaiter.OnComplete(() => // ThreadPool.QueueUserWorkItem(this.MoveNext)); return; case 0: Console.WriteLine("Foo", this.b); builder.SetResult(); return; } } } static class Program { static Task Foo(int a, int b) { var builder = AsyncTaskMethodBuilder.Create(); var stateMachine = new Foo__0(); // initialize state machine fields stateMachine.a = a; stateMachine.b = b; stateMachine.state = -1; stateMachine.builder = builder; builder.Start(ref stateMachine); // ^ similar to: ThreadPool.QueueUserWorkItem(this.MoveNext); return builder.Task; } }
WHAT YOU SEE WHAT YOU DON’T SEE static class Program { static async Task Foo(int a, int b) { Console.WriteLine("Foo", a); await Task.Yield(); Console.WriteLine("Foo", b); } } struct Foo__0 : IStateMachine { AsyncTaskMethodBuilder builder; int state; int a; int b; public void MoveNext() { switch (this.state) { case -1: Console.WriteLine("Foo", this.a); var awaiter = Taks.Yield().GetAwaiter(); this.state = 0; this.builder.AwaitUnsafeOnCompleted(ref awaiter, ref this); // ^ awaiter.OnComplete(() => // ThreadPool.QueueUserWorkItem(this.MoveNext)); return; case 0: Console.WriteLine("Foo", this.b); builder.SetResult(); return; } } } static class Program { static Task Foo(int a, int b) { var builder = AsyncTaskMethodBuilder.Create(); var stateMachine = new Foo__0(); // initialize state machine fields stateMachine.a = a; stateMachine.b = b; stateMachine.state = -1; stateMachine.builder = builder; builder.Start(ref stateMachine); // ^ similar to: ThreadPool.QueueUserWorkItem(this.MoveNext); return builder.Task; } }
WHAT YOU SEE WHAT YOU DON’T SEE static class Program { static async Task Foo(int a, int b) { Console.WriteLine("Foo", a); await Task.Yield(); Console.WriteLine("Foo", b); } } struct Foo__0 : IStateMachine { AsyncTaskMethodBuilder builder; int state; int a; int b; public void MoveNext() { switch (this.state) { case -1: Console.WriteLine("Foo", this.a); var awaiter = Taks.Yield().GetAwaiter(); this.state = 0; this.builder.AwaitUnsafeOnCompleted(ref awaiter, ref this); // ^ awaiter.OnComplete(() => // ThreadPool.QueueUserWorkItem(this.MoveNext)); return; case 0: Console.WriteLine("Foo", this.b); builder.SetResult(); return; } } } static class Program { static Task Foo(int a, int b) { var builder = AsyncTaskMethodBuilder.Create(); var stateMachine = new Foo__0(); // initialize state machine fields stateMachine.a = a; stateMachine.b = b; stateMachine.state = -1; stateMachine.builder = builder; builder.Start(ref stateMachine); // ^ similar to: ThreadPool.QueueUserWorkItem(this.MoveNext); return builder.Task; } }
WHAT YOU SEE WHAT YOU DON’T SEE static class Program { static async Task Foo(int a, int b) { Console.WriteLine("Foo", a); await Task.Yield(); Console.WriteLine("Foo", b); } } struct Foo__0 : IStateMachine { AsyncTaskMethodBuilder builder; int state; int a; int b; public void MoveNext() { switch (this.state) { case -1: Console.WriteLine("Foo", this.a); var awaiter = Taks.Yield().GetAwaiter(); this.state = 0; this.builder.AwaitUnsafeOnCompleted(ref awaiter, ref this); // ^ awaiter.OnComplete(() => // ThreadPool.QueueUserWorkItem(this.MoveNext)); return; case 0: Console.WriteLine("Foo", this.b); builder.SetResult(); return; } } } static class Program { static Task Foo(int a, int b) { var builder = AsyncTaskMethodBuilder.Create(); var stateMachine = new Foo__0(); // initialize state machine fields stateMachine.a = a; stateMachine.b = b; stateMachine.state = -1; stateMachine.builder = builder; builder.Start(ref stateMachine); // ^ similar to: ThreadPool.QueueUserWorkItem(this.MoveNext); return builder.Task; } }
AWAITER PATTERN Custom awaitable objects public readonly struct PromiseAwaiter<T> : INotifyCompletion { private readonly Promise<T> promise; public PromiseAwaiter(Promise<T> promise) { this.promise = promise; } #region mandatory awaiter methods public bool IsCompleted => promise.IsCompleted; public T GetResult() => promise.Result; public void OnCompleted(Action continuation) => promise.RegisterContinuation(continuation); #endregion }
ASYNC METHOD BUILDER Custom async methods public struct PromiseAsyncMethodBuilder<T> { private Promise<T>? promise; #region mandatory methods for async state machine builder public static PromiseAsyncMethodBuilder<T> Create() => default; public Promise<T> Task => promise ??= new Promise<T>(); public void SetException(Exception e) => Task.TrySetException(e); public void SetResult(T result) => Task.TrySetResult(result); public void AwaitOnCompleted<TAwaiter, TStateMachine>(ref TAwaiter awaiter, ref TStateMachine stateMachine) where TAwaiter : INotifyCompletion where TStateMachine : IAsyncStateMachine { } public void AwaitUnsafeOnCompleted<TAwaiter, TStateMachine>(ref TAwaiter awaiter, ref TStateMachine stateMachine) where TAwaiter : ICriticalNotifyCompletion where TStateMachine : IAsyncStateMachine { } public void Start<TStateMachine>(ref TStateMachine stateMachine) where TStateMachine : IAsyncStateMachine { } public void SetStateMachine(IAsyncStateMachine stateMachine) { } #endregion }
DEMO Custom async method builder
SUMMARY
 Custom AsyncMethodBuilder “docs”: https://github.com/dotnet/roslyn/blob/master/docs/features/task-types.md  Building custom (affine) thread pool: https://bartoszsypytkowski.com/thread-safety-with-affine-thread-pools/  How Go scheduler works: https://www.youtube.com/watch?v=-K11rY57K7k REFERENCES
THANK YOU
Behind modern concurrency primitives
REFERENCES B1 IMemoryOwner.Memory (disposable) B2 IMemoryOwner.Memory (disposable) B3 IMemoryOwner.Memory (disposable) F1 Message frame (disposable) F2 Message frame (disposable)Delimeter

More Related Content

What's hot (20)

PDF
From Zero to Application Delivery with NixOS
Susan Potter
 
PDF
Security and dev ops for high velocity organizations
Chef
 
PDF
Forgive me for i have allocated
Tomasz Kowalczewski
 
PPTX
psCloudstack Internals
Hans van Veen
 
PDF
Ansible not only for Dummies
Łukasz Proszek
 
PDF
Testing your infrastructure with litmus
Bram Vogelaar
 
PDF
Event loop
codepitbull
 
PDF
PyCon AU 2015 - Using benchmarks to understand how wsgi servers work
Graham Dumpleton
 
PDF
PyCon HK 2015 - Monitoring the performance of python web applications
Graham Dumpleton
 
PDF
Introduction to Ansible (Pycon7 2016)
Ivan Rossi
 
PDF
Using Node.js to Build Great Streaming Services - HTML5 Dev Conf
Tom Croucher
 
PDF
Take control of your Jenkins jobs via job DSL.
Łukasz Proszek
 
KEY
PyCon AU 2012 - Debugging Live Python Web Applications
Graham Dumpleton
 
PDF
Vagrant for real (codemotion rome 2016)
Michele Orselli
 
PDF
An Introduction to Twisted
sdsern
 
PDF
Loophole: Timing Attacks on Shared Event Loops in Chrome
cgvwzq
 
PDF
Celery
Òscar Vilaplana
 
PDF
Celery - A Distributed Task Queue
Duy Do
 
PDF
Node.js: Continuation-Local-Storage and the Magic of AsyncListener
Islam Sharabash
 
PDF
KubeCon EU 2016: Getting the Jobs Done With Kubernetes
KubeAcademy
 
From Zero to Application Delivery with NixOS
Susan Potter
 
Security and dev ops for high velocity organizations
Chef
 
Forgive me for i have allocated
Tomasz Kowalczewski
 
psCloudstack Internals
Hans van Veen
 
Ansible not only for Dummies
Łukasz Proszek
 
Testing your infrastructure with litmus
Bram Vogelaar
 
Event loop
codepitbull
 
PyCon AU 2015 - Using benchmarks to understand how wsgi servers work
Graham Dumpleton
 
PyCon HK 2015 - Monitoring the performance of python web applications
Graham Dumpleton
 
Introduction to Ansible (Pycon7 2016)
Ivan Rossi
 
Using Node.js to Build Great Streaming Services - HTML5 Dev Conf
Tom Croucher
 
Take control of your Jenkins jobs via job DSL.
Łukasz Proszek
 
PyCon AU 2012 - Debugging Live Python Web Applications
Graham Dumpleton
 
Vagrant for real (codemotion rome 2016)
Michele Orselli
 
An Introduction to Twisted
sdsern
 
Loophole: Timing Attacks on Shared Event Loops in Chrome
cgvwzq
 
Celery
Òscar Vilaplana
 
Celery - A Distributed Task Queue
Duy Do
 
Node.js: Continuation-Local-Storage and the Magic of AsyncListener
Islam Sharabash
 
KubeCon EU 2016: Getting the Jobs Done With Kubernetes
KubeAcademy
 

Similar to Behind modern concurrency primitives (20)

PPT
Server side JavaScript: going all the way
Oleg Podsechin
 
PPT
JS everywhere 2011
Oleg Podsechin
 
PPTX
Binary Studio Academy: Concurrency in C# 5.0
Binary Studio
 
PPTX
Avoiding Callback Hell with Async.js
cacois
 
PPTX
[NDC 2019] Enterprise-Grade Serverless
KatyShimizu
 
PPTX
[NDC 2019] Functions 2.0: Enterprise-Grade Serverless
KatyShimizu
 
PPTX
Async programming and python
Chetan Giridhar
 
KEY
Writing robust Node.js applications
Tom Croucher
 
PDF
CI Provisioning with OpenStack - Gidi Samuels - OpenStack Day Israel 2016
Cloud Native Day Tel Aviv
 
PDF
Play Framework
Harinath Krishnamoorthy
 
PDF
What the CRaC - Superfast JVM startup
Gerrit Grunwald
 
PDF
Container orchestration from theory to practice
Docker, Inc.
 
PDF
Release with confidence
John Congdon
 
PPTX
Copper: A high performance workflow engine
dmoebius
 
PPTX
Real World Lessons on the Pain Points of Node.js Applications
Ben Hall
 
PDF
Seastar @ NYCC++UG
Avi Kivity
 
PPTX
Alexey Orlenko ''High-performance IPC and RPC for microservices and apps''
OdessaJS Conf
 
KEY
fog or: How I Learned to Stop Worrying and Love the Cloud (OpenStack Edition)
Wesley Beary
 
PDF
Streaming Dataflow with Apache Flink
huguk
 
PPTX
CTU June 2011 - C# 5.0 - ASYNC & Await
Spiffy
 
Server side JavaScript: going all the way
Oleg Podsechin
 
JS everywhere 2011
Oleg Podsechin
 
Binary Studio Academy: Concurrency in C# 5.0
Binary Studio
 
Avoiding Callback Hell with Async.js
cacois
 
[NDC 2019] Enterprise-Grade Serverless
KatyShimizu
 
[NDC 2019] Functions 2.0: Enterprise-Grade Serverless
KatyShimizu
 
Async programming and python
Chetan Giridhar
 
Writing robust Node.js applications
Tom Croucher
 
CI Provisioning with OpenStack - Gidi Samuels - OpenStack Day Israel 2016
Cloud Native Day Tel Aviv
 
Play Framework
Harinath Krishnamoorthy
 
What the CRaC - Superfast JVM startup
Gerrit Grunwald
 
Container orchestration from theory to practice
Docker, Inc.
 
Release with confidence
John Congdon
 
Copper: A high performance workflow engine
dmoebius
 
Real World Lessons on the Pain Points of Node.js Applications
Ben Hall
 
Seastar @ NYCC++UG
Avi Kivity
 
Alexey Orlenko ''High-performance IPC and RPC for microservices and apps''
OdessaJS Conf
 
fog or: How I Learned to Stop Worrying and Love the Cloud (OpenStack Edition)
Wesley Beary
 
Streaming Dataflow with Apache Flink
huguk
 
CTU June 2011 - C# 5.0 - ASYNC & Await
Spiffy
 
Ad

More from Bartosz Sypytkowski (15)

PPTX
Full text search, vector search or both?
Bartosz Sypytkowski
 
PPTX
Service-less communication: is it possible?
Bartosz Sypytkowski
 
PPTX
Serviceless or how to build software without servers
Bartosz Sypytkowski
 
PPTX
Postgres indexes: how to make them work for your application
Bartosz Sypytkowski
 
PPTX
How do databases perform live backups and point-in-time recovery
Bartosz Sypytkowski
 
PPTX
Scaling connections in peer-to-peer applications
Bartosz Sypytkowski
 
PPTX
Rich collaborative data structures for everyone
Bartosz Sypytkowski
 
PPTX
Postgres indexes
Bartosz Sypytkowski
 
PPTX
Collaborative eventsourcing
Bartosz Sypytkowski
 
PPTX
Living in eventually consistent reality
Bartosz Sypytkowski
 
PPTX
Virtual machines - how they work
Bartosz Sypytkowski
 
PPTX
Short story of time
Bartosz Sypytkowski
 
PPTX
Collaborative text editing
Bartosz Sypytkowski
 
PPTX
The last mile from db to disk
Bartosz Sypytkowski
 
PPTX
GraphQL - an elegant weapon... for more civilized age
Bartosz Sypytkowski
 
Full text search, vector search or both?
Bartosz Sypytkowski
 
Service-less communication: is it possible?
Bartosz Sypytkowski
 
Serviceless or how to build software without servers
Bartosz Sypytkowski
 
Postgres indexes: how to make them work for your application
Bartosz Sypytkowski
 
How do databases perform live backups and point-in-time recovery
Bartosz Sypytkowski
 
Scaling connections in peer-to-peer applications
Bartosz Sypytkowski
 
Rich collaborative data structures for everyone
Bartosz Sypytkowski
 
Postgres indexes
Bartosz Sypytkowski
 
Collaborative eventsourcing
Bartosz Sypytkowski
 
Living in eventually consistent reality
Bartosz Sypytkowski
 
Virtual machines - how they work
Bartosz Sypytkowski
 
Short story of time
Bartosz Sypytkowski
 
Collaborative text editing
Bartosz Sypytkowski
 
The last mile from db to disk
Bartosz Sypytkowski
 
GraphQL - an elegant weapon... for more civilized age
Bartosz Sypytkowski
 
Ad

Recently uploaded (20)

PPTX
Building and Operating a Private Cloud with CloudStack and LINBIT CloudStack ...
ShapeBlue
 
PPTX
python advanced data structure dictionary with examples python advanced data ...
sprasanna11
 
PPTX
TYPES OF COMMUNICATION Presentation of ICT
JulieBinwag
 
PDF
Generative AI in Healthcare: Benefits, Use Cases & Challenges
Lily Clark
 
PDF
Women in Automation Presents: Reinventing Yourself — Bold Career Pivots That ...
DianaGray10
 
PDF
How a Code Plagiarism Checker Protects Originality in Programming
Code Quiry
 
PDF
CloudStack GPU Integration - Rohit Yadav
ShapeBlue
 
PDF
"Effect, Fiber & Schema: tactical and technical characteristics of Effect.ts"...
Fwdays
 
PPTX
Darren Mills The Migration Modernization Balancing Act: Navigating Risks and...
AWS Chicago
 
PDF
The Past, Present & Future of Kenya's Digital Transformation
Moses Kemibaro
 
PDF
Shuen Mei Parth Sharma Boost Productivity, Innovation and Efficiency wit...
AWS Chicago
 
PDF
NewMind AI Journal - Weekly Chronicles - July'25 Week II
NewMind AI
 
PDF
Julia Furst Morgado The Lazy Guide to Kubernetes with EKS Auto Mode + Karpenter
AWS Chicago
 
PDF
Bitcoin+ Escalando sin concesiones - Parte 1
Fernando Paredes García
 
PDF
Empowering Cloud Providers with Apache CloudStack and Stackbill
ShapeBlue
 
PPTX
Lecture 5 - Agentic AI and model context protocol.pptx
Dr. LAM Yat-fai (林日辉)
 
PPTX
Extensions Framework (XaaS) - Enabling Orchestrate Anything
ShapeBlue
 
PPTX
Simplifying End-to-End Apache CloudStack Deployment with a Web-Based Automati...
ShapeBlue
 
PDF
UiPath vs Other Automation Tools Meeting Presentation.pdf
Tracy Dixon
 
PDF
Market Insight : ETH Dominance Returns
CIFDAQ
 
Building and Operating a Private Cloud with CloudStack and LINBIT CloudStack ...
ShapeBlue
 
python advanced data structure dictionary with examples python advanced data ...
sprasanna11
 
TYPES OF COMMUNICATION Presentation of ICT
JulieBinwag
 
Generative AI in Healthcare: Benefits, Use Cases & Challenges
Lily Clark
 
Women in Automation Presents: Reinventing Yourself — Bold Career Pivots That ...
DianaGray10
 
How a Code Plagiarism Checker Protects Originality in Programming
Code Quiry
 
CloudStack GPU Integration - Rohit Yadav
ShapeBlue
 
"Effect, Fiber & Schema: tactical and technical characteristics of Effect.ts"...
Fwdays
 
Darren Mills The Migration Modernization Balancing Act: Navigating Risks and...
AWS Chicago
 
The Past, Present & Future of Kenya's Digital Transformation
Moses Kemibaro
 
Shuen Mei Parth Sharma Boost Productivity, Innovation and Efficiency wit...
AWS Chicago
 
NewMind AI Journal - Weekly Chronicles - July'25 Week II
NewMind AI
 
Julia Furst Morgado The Lazy Guide to Kubernetes with EKS Auto Mode + Karpenter
AWS Chicago
 
Bitcoin+ Escalando sin concesiones - Parte 1
Fernando Paredes García
 
Empowering Cloud Providers with Apache CloudStack and Stackbill
ShapeBlue
 
Lecture 5 - Agentic AI and model context protocol.pptx
Dr. LAM Yat-fai (林日辉)
 
Extensions Framework (XaaS) - Enabling Orchestrate Anything
ShapeBlue
 
Simplifying End-to-End Apache CloudStack Deployment with a Web-Based Automati...
ShapeBlue
 
UiPath vs Other Automation Tools Meeting Presentation.pdf
Tracy Dixon
 
Market Insight : ETH Dominance Returns
CIFDAQ
 

Behind modern concurrency primitives

  • 3.  Concurrency in user vs. kernel space  Thread pools  Schedulers  Coroutines  State machines AGENDA
  • 4. WHY DO WE USE THIS?
  • 5. ASYNC/AWAIT Task.Run(async () => { await using var conn = new SqlConnection(ConnectionString); await conn.OpenAsync(); var user = await conn.QueryFirstAsync<User>( "SELECT * FROM Users WHERE Id = @id", new { id = 100 }); Console.WriteLine($"Received user {user.FirstName} {user.LastName}"); });
  • 7. THREAD API new Thread(() => { var conn = new SqlConnection(ConnectionString); conn.Open(); var user = conn.QueryFirst<User>( "SELECT * FROM Users WHERE Id = @id", new { id = 100 }); Console.WriteLine($"Received user {user.FirstName} {user.LastName}"); }).Start();
  • 8. Thread Pool* Managed Thread Managed Thread OS Thread THREADS TASKS User space Kernel space User space Kernel space Managed Thread OS Thread OS Thread Scheduler OS Thread OS Thread OS Thread Task Task Task Task Task Task Task Task Task Task Task Task Task Task Task Task Task Task
  • 10. THREAD POOL BASICS CPU 1 Agent Managed Thread CPU 2 Agent Managed Thread CPU 3 Agent Managed Thread CPU 4 Agent Managed Thread Job Queue
  • 11. HARDWARE ARCHITECTURE 101 CPU 1 L1 cache L1 cache L3 cache CPU 2 L1 cache L1 cache RAM CPU L1 cache: 1ns CPU L2 cache: 4-7ns CPU L2 cache: 100ns
  • 12. PINNING THREAD TO CPU var threads = Process.GetCurrentProcess().Threads; var cpuCount = Environment.ProcessorCount; Console.WriteLine($"Using {threads.Count} threads on {cpuCount} cores."); // => Using 10 threads on 4 cores. for (int i = 0; i < threads.Count; i++) { var pthread = threads[i]; var cpuMask = (1L << (i % cpuCount)); pthread.IdealProcessor = i & cpuCount; // set preferred thread(s) pthread.ProcessorAffinity = (IntPtr)cpuMask; // set thread affinity mask } NOTE: it’s not always possible (iOS) or respected.
  • 13. HARDWARE ARCHITECTURE NUMA CPU 1 CPU 2 L1-2cache CPU 3 CPU 4 L1-2cache CPU 5 CPU 6 L1-2cache CPU 7 CPU 8 L1-2cache
  • 15. THREAD POOL WORK STEALING CPU 1 Agent Managed Thread CPU 2 Agent Managed Thread CPU 3 Agent Managed Thread CPU 4 Agent Managed Thread Job Job Job Job Job Job Job Job Job Job
  • 16. THREAD POOL WORK STEALING CPU 1 Agent Managed Thread CPU 2 Agent Managed Thread CPU 3 Agent Managed Thread CPU 4 Agent Managed Thread Job Job Job Job Job Job ? Job Job Job Empty Queue
  • 17. THREAD POOL WORK STEALING CPU 1 Agent Managed Thread CPU 2 Agent Managed Thread CPU 3 Agent Managed Thread CPU 4 Agent Managed Thread Job Job Job Job Job Job ? Job Job Job
  • 18. THREAD POOL WORK STEALING CPU 1 Agent Managed Thread CPU 2 Agent Managed Thread CPU 3 Agent Managed Thread CPU 4 Agent Managed Thread Job Job Job Job Job Job Job JobJob
  • 19. COMPAR ING VECTOR CLOCKS AFFINITY BASED THREAD POOL CPU 1 Agent Managed Thread CPU 2 Agent Managed Thread CPU 3 Agent Managed Thread CPU 4 Agent Managed Thread S1 S1 S1 S1 S2 S2 S3 S2 S3 S3 S4 S5 S4 S4 S5 Always map entity to the same thread / core
  • 20. AFFINITY THREAD POOL: PERFORMANCE Source: https://scalac.io/improving-akka-dispatchers/
  • 22. WHY IS THIS CODE BAD? static async Task ProcessFiles(FileInfo[] files) { var tasks = new Task[files.Length]; for (int i = 0; i < files.Length; i++) tasks[i] = ProcessFile(files[i]); await Task.WhenAll(tasks); } static async Task ProcessFile(FileInfo file) { using var stream = file.OpenRead(); using var reader = new StreamReader(stream); var text = reader.ReadToEnd(); Process(text); }
  • 23. WHY IS THIS CODE BAD? static async Task ProcessFiles(FileInfo[] files) { var tasks = new Task[files.Length]; for (int i = 0; i < files.Length; i++) tasks[i] = ProcessFile(files[i]); await Task.WhenAll(tasks); } static async Task ProcessFile(FileInfo file) { using var stream = file.OpenRead(); using var reader = new StreamReader(stream); var text = reader.ReadToEnd(); Process(text); } Blocking the thread belonging to a thread pool shared with other tasks.
  • 24. I/O 1. Just use async I/O API… WHAT CAN WE DO ABOUT IT?
  • 25. I/O 1. Just use async I/O API… 2. … but what when it’s not possible? WHAT CAN WE DO ABOUT IT?
  • 26. I/O 1. Just use async I/O API… 2. … but what when it’s not possible? WHAT CAN WE DO ABOUT IT? internal static class LmdbMethods { [DllImport("lmdb", CallingConvention = CallingConvention.Cdecl)] public static extern int mdb_dbi_open(IntPtr txn, string name, DatabaseOpenFlags flags, out uint db); [DllImport("lmdb", CallingConvention = CallingConvention.Cdecl)] public static extern int mdb_cursor_get(IntPtr cursor, ref ValueStructure key, ref ValueStructure data, CursorOperation op); [DllImport("lmdb", CallingConvention = CallingConvention.Cdecl)] public static extern int mdb_cursor_put(IntPtr cursor, ref ValueStructure key, ref ValueStructure value, CursorPutOptions flags); // other methods }
  • 27. THREAD POOL I/O THREADS CPU 1 Agent Managed Thread CPU 2 Agent Managed Thread CPU 3 Agent Managed Thread CPU 4 Agent Managed Thread Managed Thread I/O Threads Managed Thread Managed Thread
  • 28. THREAD POOL I/O THREADS CPU 1 Agent Managed Thread CPU 2 Agent Managed Thread CPU 3 Agent Managed Thread CPU 4 Agent Managed Thread Managed Thread I/O Threads Managed Thread Managed Thread J1
  • 29. THREAD POOL I/O THREADS CPU 1 Agent Managed Thread CPU 2 Agent Managed Thread CPU 3 Agent Managed Thread CPU 4 Agent Managed Thread Managed Thread I/O Threads Managed Thread Managed Thread J1
  • 30. THREAD POOL I/O THREADS CPU 1 Agent Managed Thread CPU 2 Agent Managed Thread CPU 3 Agent Managed Thread CPU 4 Agent Managed Thread Managed Thread I/O Threads Managed Thread Managed Thread J1
  • 31. THREAD POOL I/O THREADS CPU 1 Agent Managed Thread CPU 2 Agent Managed Thread CPU 3 Agent Managed Thread CPU 4 Agent Managed Thread Managed Thread I/O Threads Managed Thread Managed Thread J2
  • 32. GOROUTINES & I/O DEALING WITH KERNEL CALLS
  • 33. COMPAR ING VECTOR CLOCKS GOROUTINES I/O CALLS CPU 1 Agent Runtime Thread CPU 2 Agent Runtime Thread CPU 3 Agent Runtime Thread CPU 4 Agent Runtime Thread
  • 34. COMPAR ING VECTOR CLOCKS GOROUTINES I/O CALLS CPU 1 Agent Runtime Thread CPU 2 Agent Runtime Thread CPU 3 Agent Runtime Thread CPU 4 Agent Runtime Thread sys call
  • 35. COMPAR ING VECTOR CLOCKS GOROUTINES I/O CALLS CPU 1 Agent CPU 2 Agent Runtime Thread CPU 3 Agent Runtime Thread CPU 4 Agent Runtime Thread Runtime Thread sys call
  • 36. COMPAR ING VECTOR CLOCKS GOROUTINES I/O CALLS CPU 1 Agent CPU 2 Agent Runtime Thread CPU 3 Agent Runtime Thread CPU 4 Agent Runtime Thread Runtime Thread sys call Runtime Thread
  • 38. Scheduler depends on coroutines to return control to scheduler. PREEMPTIVE Scheduler is in charge of execution. Coroutines can be preempted. COOPERATIVE
  • 39. SCENARIO #2 CREATE AN EXCEL FILE
  • 40. COMPAR ING VECTOR CLOCKS BUILDING AN EXCEL FILE app.get('/valuations.xlsx', async (req, res) => { var workbook = new Excel.Workbook(); var worksheet = workbook.addWorksheet("Valuations"); var valuations = await getValuations(req.params.id); for (let valuation in valuations) { worksheet.addRow(valuation); } await workbook.xlsx.write(res); });
  • 41. COMPAR ING VECTOR CLOCKS BUILDING AN EXCEL FILE app.get('/valuations.xlsx', async (req, res) => { var workbook = new Excel.Workbook(); var worksheet = workbook.addWorksheet("Valuations"); var valuations = await getValuations(req.params.id); for (let valuation in valuations) { worksheet.addRow(valuation); // 500μs } await workbook.xlsx.write(res); });
  • 42. COMPAR ING VECTOR CLOCKS BUILDING AN EXCEL FILE app.get('/valuations.xlsx', async (req, res) => { var workbook = new Excel.Workbook(); var worksheet = workbook.addWorksheet("Valuations"); var valuations = await getValuations(req.params.id); for (let valuation in valuations) { // 20,000 items worksheet.addRow(valuation); // 500μs } await workbook.xlsx.write(res); });
  • 43. COMPAR ING VECTOR CLOCKS BUILDING AN EXCEL FILE app.get('/valuations.xlsx', async (req, res) => { var workbook = new Excel.Workbook(); var worksheet = workbook.addWorksheet("Valuations"); var valuations = await getValuations(req.params.id); for (let valuation in valuations) { // 20,000 items worksheet.addRow(valuation); // 500μs } await workbook.xlsx.write(res); }); Total loop execution time: 10 seconds
  • 44. HOW TO DEAL WITH LONG RUNNING TASKS?
  • 45. LONG RUNNING TASK SEPARATE THREAD CPU 1 Agent Managed Thread CPU 2 Agent Managed Thread CPU 3 Agent Managed Thread CPU 4 Agent Managed Thread Scheduler
  • 46. LONG RUNNING TASK SEPARATE THREAD CPU 1 Agent Managed Thread CPU 2 Agent Managed Thread CPU 3 Agent Managed Thread CPU 4 Agent Managed Thread Scheduler Task Task.Factory.StartNew(DownloadExcel, TaskCreationOptions.LongRunning)
  • 47. LONG RUNNING TASK SEPARATE THREAD CPU 1 Agent Managed Thread CPU 2 Agent Managed Thread CPU 3 Agent Managed Thread CPU 4 Agent Managed Thread Scheduler Task Task.Factory.StartNew(DownloadExcel, TaskCreationOptions.LongRunning) Managed Thread
  • 48. LONG RUNNING TASK SEPARATE THREAD CPU 1 Agent Managed Thread CPU 2 Agent Managed Thread CPU 3 Agent Managed Thread CPU 4 Agent Managed Thread Scheduler Managed Thread Task
  • 49. LONG RUNNING TASK SEPARATE THREAD CPU 1 Agent Managed Thread CPU 2 Agent Managed Thread CPU 3 Agent Managed Thread CPU 4 Agent Managed Thread Scheduler Managed Thread Task
  • 50. COMPAR ING VECTOR CLOCKS INTRODUCING INTERRUPTION POINTS function park() { return new Promise((resolve) => setTimeout(resolve, 0)); } app.get('/valuations.xlsx', async (req, res) => { var workbook = new Excel.Workbook(); var worksheet = workbook.addWorksheet("Valuations"); var valuations = await getValuations(req.params.id); var i = 0; for (let valuation in valuations) { // 20,000 items if ((i++) % 100 === 0) { await park(); } worksheet.addRow(valuation); // 500μs } await workbook.xlsx.write(res); });
  • 52. STEP-BASED PREEMPTION % hello world program -module(helloworld). -export([start/0, write/1]). write([]) -> ok; write([File|T]) -> io:fwrite("Writing a file ~p~n", File), write(T). start() -> Files = ["A", "B", "C"], write(Files).
  • 53. STEP-BASED PREEMPTION Reduction counter under the hood % hello world program -module(helloworld). -export([start/0, write/1]). write([], Reductions) -> ok; write([File|T], Reductions) -> % if Reductions == 0 then yield() io:fwrite("Writing a file ~p~n", File), write(T, Reductions-1). start() -> Files = ["A", "B", "C"], write(Files, 2000).
  • 54. • JavaScript promises • .NET Task Parallel Library • Java/Scala Futures PREEMPTIVE • OS threads • Go goroutines • Erlang processes COOPERATIVE
  • 56. let run () = async { let sw = Stopwatch() sw.Start() let t1 = Async.Sleep(5000) let t2 = Async.Sleep(5000) do! t1 do! t2 printfn "Time passed: %ims" sw.ElapsedMilliseconds } static async Task Run() { var sw = new Stopwatch(); sw.Start(); var t1 = Task.Delay(TimeSpan.FromSeconds(5)); var t2 = Task.Delay(TimeSpan.FromSeconds(5)); await t1; await t2; Console.WriteLine( $"Time passed: {sw.ElapsedMilliseconds}ms"); }
  • 57. let run () = async { let sw = Stopwatch() sw.Start() let t1 = Async.Sleep(5000) let t2 = Async.Sleep(5000) do! t1 do! t2 printfn "Time passed: %ims" sw.ElapsedMilliseconds } static async Task Run() { var sw = new Stopwatch(); sw.Start(); var t1 = Task.Delay(TimeSpan.FromSeconds(5)); var t2 = Task.Delay(TimeSpan.FromSeconds(5)); await t1; await t2; Console.WriteLine( $"Time passed: {sw.ElapsedMilliseconds}ms"); } EAGER LAZY
  • 59. • JavaScript promises • .NET Task Parallel Library • Java/Scala Futures STACKFUL • Go goroutines • LUA coroutines • (soon) Java Loom project STACKLESS
  • 60. COMPAR ING VECTOR CLOCKS STACKLESS COROUTINES OLD CALLBACK API app.get('/valuations.xlsx', function (req, res, next) { var workbook = new Excel.Workbook(); var worksheet = workbook.addWorksheet("Valuations"); getValuations(req.params.id, function (err, valuations) { for (let valuation in valuations) { worksheet.addRow(valuation); } workbook.xlsx.write(res, function (err) { next(); }); }); });
  • 61. COMPAR ING VECTOR CLOCKS STACKLESS COROUTINES app.get('/valuations.xlsx', (req, res) => { var workbook = new Excel.Workbook(); var worksheet = workbook.addWorksheet("Valuations"); return getValuations(req.params.id) .then((valuations) => { for (let valuation in valuations) { worksheet.addRow(valuation); } return workbook.xlsx.write(res); }); }); OLD PROMISE API
  • 62. COMPAR ING VECTOR CLOCKS STACKLESS COROUTINES app.get('/valuations.xlsx', async (req, res) => { var workbook = new Excel.Workbook(); var worksheet = workbook.addWorksheet("Valuations"); var valuations = await getValuations(req.params.id); for (let valuation in valuations) { worksheet.addRow(valuation); } await workbook.xlsx.write(res); });
  • 63. PROBLEM OF FUNCTION COLOURING
  • 64. STACKFUL COROUTINES STACK main activation frame function foo(a, b) print("foo", a) bar("foo", b) end function bar(caller, x) print(“bar: ", caller) coroutine.yield() print(“bar", x) end co = coroutine.create(foo) coroutine.resume(co, 1, 2) print(“main”) coroutine.resume(co)
  • 65. STACKFUL COROUTINES STACK main activation frame function foo(a, b) print("foo", a) bar("foo", b) end function bar(caller, x) print(“bar: ", caller) coroutine.yield() print(“bar", x) end co = coroutine.create(foo) coroutine.resume(co, 1, 2) print(“main”) coroutine.resume(co) foo activation frame Start of coroutine stack
  • 66. STACKFUL COROUTINES STACK main activation frame function foo(a, b) print("foo", a) bar("foo", b) end function bar(caller, x) print(“bar: ", caller) coroutine.yield() print(“bar", x) end co = coroutine.create(foo) coroutine.resume(co, 1, 2) print(“main”) coroutine.resume(co) foo activation frame print activation frame
  • 67. STACKFUL COROUTINES STACK main activation frame function foo(a, b) print("foo", a) bar("foo", b) end function bar(caller, x) print(“bar: ", caller) coroutine.yield() print(“bar", x) end co = coroutine.create(foo) coroutine.resume(co, 1, 2) print(“main”) coroutine.resume(co) foo activation frame bar activation frame
  • 68. STACKFUL COROUTINES STACK main activation frame function foo(a, b) print("foo", a) bar("foo", b) end function bar(caller, x) print(“bar: ", caller) coroutine.yield() print(“bar", x) end co = coroutine.create(foo) coroutine.resume(co, 1, 2) print(“main”) coroutine.resume(co) foo activation frame bar activation frame print activation frame
  • 69. STACKFUL COROUTINES STACK main activation frame function foo(a, b) print("foo", a) bar("foo", b) end function bar(caller, x) print(“bar: ", caller) coroutine.yield() print(“bar", x) end co = coroutine.create(foo) coroutine.resume(co, 1, 2) print(“main”) coroutine.resume(co) foo activation frame bar activation frame Park coroutine stack
  • 70. STACKFUL COROUTINES STACK main activation frame function foo(a, b) print("foo", a) bar("foo", b) end function bar(caller, x) print(“bar: ", caller) coroutine.yield() print(“bar", x) end co = coroutine.create(foo) coroutine.resume(co, 1, 2) print(“main”) coroutine.resume(co) foo activation frame bar activation frame Park coroutine stack
  • 71. STACKFUL COROUTINES STACK main activation frame function foo(a, b) print("foo", a) bar("foo", b) end function bar(caller, x) print(“bar: ", caller) coroutine.yield() print(“bar", x) end co = coroutine.create(foo) coroutine.resume(co, 1, 2) print(“main”) coroutine.resume(co) foo activation frame bar activation frame
  • 72. STACKFUL COROUTINES STACK main activation frame function foo(a, b) print("foo", a) bar("foo", b) end function bar(caller, x) print(“bar: ", caller) coroutine.yield() print(“bar", x) end co = coroutine.create(foo) coroutine.resume(co, 1, 2) print(“main”) coroutine.resume(co) foo activation frame bar activation frame print activation frame
  • 73. STACKFUL COROUTINES STACK main activation frame function foo(a, b) print("foo", a) bar("foo", b) end function bar(caller, x) print(“bar: ", caller) coroutine.yield() print(“bar", x) end co = coroutine.create(foo) coroutine.resume(co, 1, 2) print(“main”) coroutine.resume(co) foo activation frame bar activation frame Resume coroutine stack
  • 74. STACKFUL COROUTINES STACK main activation frame function foo(a, b) print("foo", a) bar("foo", b) end function bar(caller, x) print(“bar: ", caller) coroutine.yield() print(“bar", x) end co = coroutine.create(foo) coroutine.resume(co, 1, 2) print(“main”) coroutine.resume(co) foo activation frame bar activation frame
  • 75. STACKFUL COROUTINES STACK main activation frame function foo(a, b) print("foo", a) bar("foo", b) end function bar(caller, x) print(“bar: ", caller) coroutine.yield() print(“bar", x) end co = coroutine.create(foo) coroutine.resume(co, 1, 2) print(“main”) coroutine.resume(co) foo activation frame bar activation frame print activation frame
  • 76. WHAT IS THE CORRECT STACK SIZE FOR EACH COROUTINE?
  • 78. ASYNC/AWAIT UNITY COROUTINES IEnumerable WaitAndPrint() { print("Starting at " + Time.time); yield return new WaitForSeconds(0.1f); print("Ending at " + Time.time); } async Task WaitAndPrint() { Console.WriteLine("Starting at " + DateTime.Now); await Task.Delay(100); Console.WriteLine("Ending at " + DateTime.Now); }
  • 81. WHAT YOU SEE WHAT YOU DON’T SEE static class Program { static async Task Foo(int a, int b) { Console.WriteLine("Foo", a); await Task.Yield(); Console.WriteLine("Foo", b); } } struct Foo__0 : IStateMachine { AsyncTaskMethodBuilder builder; int state; int a; int b; public void MoveNext() { switch (this.state) { case -1: Console.WriteLine("Foo", this.a); var awaiter = Taks.Yield().GetAwaiter(); this.state = 0; this.builder.AwaitUnsafeOnCompleted(ref awaiter, ref this); // ^ awaiter.OnComplete(() => // ThreadPool.QueueUserWorkItem(this.MoveNext)); return; case 0: Console.WriteLine("Foo", this.b); builder.SetResult(); return; } } } static class Program { static Task Foo(int a, int b) { var builder = AsyncTaskMethodBuilder.Create(); var stateMachine = new Foo__0(); // initialize state machine fields stateMachine.a = a; stateMachine.b = b; stateMachine.state = -1; stateMachine.builder = builder; builder.Start(ref stateMachine); // ^ similar to: ThreadPool.QueueUserWorkItem(this.MoveNext); return builder.Task; } }
  • 82. WHAT YOU SEE WHAT YOU DON’T SEE static class Program { static async Task Foo(int a, int b) { Console.WriteLine("Foo", a); await Task.Yield(); Console.WriteLine("Foo", b); } } struct Foo__0 : IStateMachine { AsyncTaskMethodBuilder builder; int state; int a; int b; public void MoveNext() { switch (this.state) { case -1: Console.WriteLine("Foo", this.a); var awaiter = Taks.Yield().GetAwaiter(); this.state = 0; this.builder.AwaitUnsafeOnCompleted(ref awaiter, ref this); // ^ awaiter.OnComplete(() => // ThreadPool.QueueUserWorkItem(this.MoveNext)); return; case 0: Console.WriteLine("Foo", this.b); builder.SetResult(); return; } } } static class Program { static Task Foo(int a, int b) { var builder = AsyncTaskMethodBuilder.Create(); var stateMachine = new Foo__0(); // initialize state machine fields stateMachine.a = a; stateMachine.b = b; stateMachine.state = -1; stateMachine.builder = builder; builder.Start(ref stateMachine); // ^ similar to: ThreadPool.QueueUserWorkItem(this.MoveNext); return builder.Task; } }
  • 83. WHAT YOU SEE WHAT YOU DON’T SEE static class Program { static async Task Foo(int a, int b) { Console.WriteLine("Foo", a); await Task.Yield(); Console.WriteLine("Foo", b); } } struct Foo__0 : IStateMachine { AsyncTaskMethodBuilder builder; int state; int a; int b; public void MoveNext() { switch (this.state) { case -1: Console.WriteLine("Foo", this.a); var awaiter = Taks.Yield().GetAwaiter(); this.state = 0; this.builder.AwaitUnsafeOnCompleted(ref awaiter, ref this); // ^ awaiter.OnComplete(() => // ThreadPool.QueueUserWorkItem(this.MoveNext)); return; case 0: Console.WriteLine("Foo", this.b); builder.SetResult(); return; } } } static class Program { static Task Foo(int a, int b) { var builder = AsyncTaskMethodBuilder.Create(); var stateMachine = new Foo__0(); // initialize state machine fields stateMachine.a = a; stateMachine.b = b; stateMachine.state = -1; stateMachine.builder = builder; builder.Start(ref stateMachine); // ^ similar to: ThreadPool.QueueUserWorkItem(this.MoveNext); return builder.Task; } }
  • 84. WHAT YOU SEE WHAT YOU DON’T SEE static class Program { static async Task Foo(int a, int b) { Console.WriteLine("Foo", a); await Task.Yield(); Console.WriteLine("Foo", b); } } struct Foo__0 : IStateMachine { AsyncTaskMethodBuilder builder; int state; int a; int b; public void MoveNext() { switch (this.state) { case -1: Console.WriteLine("Foo", this.a); var awaiter = Taks.Yield().GetAwaiter(); this.state = 0; this.builder.AwaitUnsafeOnCompleted(ref awaiter, ref this); // ^ awaiter.OnComplete(() => // ThreadPool.QueueUserWorkItem(this.MoveNext)); return; case 0: Console.WriteLine("Foo", this.b); builder.SetResult(); return; } } } static class Program { static Task Foo(int a, int b) { var builder = AsyncTaskMethodBuilder.Create(); var stateMachine = new Foo__0(); // initialize state machine fields stateMachine.a = a; stateMachine.b = b; stateMachine.state = -1; stateMachine.builder = builder; builder.Start(ref stateMachine); // ^ similar to: ThreadPool.QueueUserWorkItem(this.MoveNext); return builder.Task; } }
  • 85. WHAT YOU SEE WHAT YOU DON’T SEE static class Program { static async Task Foo(int a, int b) { Console.WriteLine("Foo", a); await Task.Yield(); Console.WriteLine("Foo", b); } } struct Foo__0 : IStateMachine { AsyncTaskMethodBuilder builder; int state; int a; int b; public void MoveNext() { switch (this.state) { case -1: Console.WriteLine("Foo", this.a); var awaiter = Taks.Yield().GetAwaiter(); this.state = 0; this.builder.AwaitUnsafeOnCompleted(ref awaiter, ref this); // ^ awaiter.OnComplete(() => // ThreadPool.QueueUserWorkItem(this.MoveNext)); return; case 0: Console.WriteLine("Foo", this.b); builder.SetResult(); return; } } } static class Program { static Task Foo(int a, int b) { var builder = AsyncTaskMethodBuilder.Create(); var stateMachine = new Foo__0(); // initialize state machine fields stateMachine.a = a; stateMachine.b = b; stateMachine.state = -1; stateMachine.builder = builder; builder.Start(ref stateMachine); // ^ similar to: ThreadPool.QueueUserWorkItem(this.MoveNext); return builder.Task; } }
  • 86. WHAT YOU SEE WHAT YOU DON’T SEE static class Program { static async Task Foo(int a, int b) { Console.WriteLine("Foo", a); await Task.Yield(); Console.WriteLine("Foo", b); } } struct Foo__0 : IStateMachine { AsyncTaskMethodBuilder builder; int state; int a; int b; public void MoveNext() { switch (this.state) { case -1: Console.WriteLine("Foo", this.a); var awaiter = Taks.Yield().GetAwaiter(); this.state = 0; this.builder.AwaitUnsafeOnCompleted(ref awaiter, ref this); // ^ awaiter.OnComplete(() => // ThreadPool.QueueUserWorkItem(this.MoveNext)); return; case 0: Console.WriteLine("Foo", this.b); builder.SetResult(); return; } } } static class Program { static Task Foo(int a, int b) { var builder = AsyncTaskMethodBuilder.Create(); var stateMachine = new Foo__0(); // initialize state machine fields stateMachine.a = a; stateMachine.b = b; stateMachine.state = -1; stateMachine.builder = builder; builder.Start(ref stateMachine); // ^ similar to: ThreadPool.QueueUserWorkItem(this.MoveNext); return builder.Task; } }
  • 87. WHAT YOU SEE WHAT YOU DON’T SEE static class Program { static async Task Foo(int a, int b) { Console.WriteLine("Foo", a); await Task.Yield(); Console.WriteLine("Foo", b); } } struct Foo__0 : IStateMachine { AsyncTaskMethodBuilder builder; int state; int a; int b; public void MoveNext() { switch (this.state) { case -1: Console.WriteLine("Foo", this.a); var awaiter = Taks.Yield().GetAwaiter(); this.state = 0; this.builder.AwaitUnsafeOnCompleted(ref awaiter, ref this); // ^ awaiter.OnComplete(() => // ThreadPool.QueueUserWorkItem(this.MoveNext)); return; case 0: Console.WriteLine("Foo", this.b); builder.SetResult(); return; } } } static class Program { static Task Foo(int a, int b) { var builder = AsyncTaskMethodBuilder.Create(); var stateMachine = new Foo__0(); // initialize state machine fields stateMachine.a = a; stateMachine.b = b; stateMachine.state = -1; stateMachine.builder = builder; builder.Start(ref stateMachine); // ^ similar to: ThreadPool.QueueUserWorkItem(this.MoveNext); return builder.Task; } }
  • 88. WHAT YOU SEE WHAT YOU DON’T SEE static class Program { static async Task Foo(int a, int b) { Console.WriteLine("Foo", a); await Task.Yield(); Console.WriteLine("Foo", b); } } struct Foo__0 : IStateMachine { AsyncTaskMethodBuilder builder; int state; int a; int b; public void MoveNext() { switch (this.state) { case -1: Console.WriteLine("Foo", this.a); var awaiter = Taks.Yield().GetAwaiter(); this.state = 0; this.builder.AwaitUnsafeOnCompleted(ref awaiter, ref this); // ^ awaiter.OnComplete(() => // ThreadPool.QueueUserWorkItem(this.MoveNext)); return; case 0: Console.WriteLine("Foo", this.b); builder.SetResult(); return; } } } static class Program { static Task Foo(int a, int b) { var builder = AsyncTaskMethodBuilder.Create(); var stateMachine = new Foo__0(); // initialize state machine fields stateMachine.a = a; stateMachine.b = b; stateMachine.state = -1; stateMachine.builder = builder; builder.Start(ref stateMachine); // ^ similar to: ThreadPool.QueueUserWorkItem(this.MoveNext); return builder.Task; } }
  • 89. WHAT YOU SEE WHAT YOU DON’T SEE static class Program { static async Task Foo(int a, int b) { Console.WriteLine("Foo", a); await Task.Yield(); Console.WriteLine("Foo", b); } } struct Foo__0 : IStateMachine { AsyncTaskMethodBuilder builder; int state; int a; int b; public void MoveNext() { switch (this.state) { case -1: Console.WriteLine("Foo", this.a); var awaiter = Taks.Yield().GetAwaiter(); this.state = 0; this.builder.AwaitUnsafeOnCompleted(ref awaiter, ref this); // ^ awaiter.OnComplete(() => // ThreadPool.QueueUserWorkItem(this.MoveNext)); return; case 0: Console.WriteLine("Foo", this.b); builder.SetResult(); return; } } } static class Program { static Task Foo(int a, int b) { var builder = AsyncTaskMethodBuilder.Create(); var stateMachine = new Foo__0(); // initialize state machine fields stateMachine.a = a; stateMachine.b = b; stateMachine.state = -1; stateMachine.builder = builder; builder.Start(ref stateMachine); // ^ similar to: ThreadPool.QueueUserWorkItem(this.MoveNext); return builder.Task; } }
  • 90. WHAT YOU SEE WHAT YOU DON’T SEE static class Program { static async Task Foo(int a, int b) { Console.WriteLine("Foo", a); await Task.Yield(); Console.WriteLine("Foo", b); } } struct Foo__0 : IStateMachine { AsyncTaskMethodBuilder builder; int state; int a; int b; public void MoveNext() { switch (this.state) { case -1: Console.WriteLine("Foo", this.a); var awaiter = Taks.Yield().GetAwaiter(); this.state = 0; this.builder.AwaitUnsafeOnCompleted(ref awaiter, ref this); // ^ awaiter.OnComplete(() => // ThreadPool.QueueUserWorkItem(this.MoveNext)); return; case 0: Console.WriteLine("Foo", this.b); builder.SetResult(); return; } } } static class Program { static Task Foo(int a, int b) { var builder = AsyncTaskMethodBuilder.Create(); var stateMachine = new Foo__0(); // initialize state machine fields stateMachine.a = a; stateMachine.b = b; stateMachine.state = -1; stateMachine.builder = builder; builder.Start(ref stateMachine); // ^ similar to: ThreadPool.QueueUserWorkItem(this.MoveNext); return builder.Task; } }
  • 91. WHAT YOU SEE WHAT YOU DON’T SEE static class Program { static async Task Foo(int a, int b) { Console.WriteLine("Foo", a); await Task.Yield(); Console.WriteLine("Foo", b); } } struct Foo__0 : IStateMachine { AsyncTaskMethodBuilder builder; int state; int a; int b; public void MoveNext() { switch (this.state) { case -1: Console.WriteLine("Foo", this.a); var awaiter = Taks.Yield().GetAwaiter(); this.state = 0; this.builder.AwaitUnsafeOnCompleted(ref awaiter, ref this); // ^ awaiter.OnComplete(() => // ThreadPool.QueueUserWorkItem(this.MoveNext)); return; case 0: Console.WriteLine("Foo", this.b); builder.SetResult(); return; } } } static class Program { static Task Foo(int a, int b) { var builder = AsyncTaskMethodBuilder.Create(); var stateMachine = new Foo__0(); // initialize state machine fields stateMachine.a = a; stateMachine.b = b; stateMachine.state = -1; stateMachine.builder = builder; builder.Start(ref stateMachine); // ^ similar to: ThreadPool.QueueUserWorkItem(this.MoveNext); return builder.Task; } }
  • 92. WHAT YOU SEE WHAT YOU DON’T SEE static class Program { static async Task Foo(int a, int b) { Console.WriteLine("Foo", a); await Task.Yield(); Console.WriteLine("Foo", b); } } struct Foo__0 : IStateMachine { AsyncTaskMethodBuilder builder; int state; int a; int b; public void MoveNext() { switch (this.state) { case -1: Console.WriteLine("Foo", this.a); var awaiter = Taks.Yield().GetAwaiter(); this.state = 0; this.builder.AwaitUnsafeOnCompleted(ref awaiter, ref this); // ^ awaiter.OnComplete(() => // ThreadPool.QueueUserWorkItem(this.MoveNext)); return; case 0: Console.WriteLine("Foo", this.b); builder.SetResult(); return; } } } static class Program { static Task Foo(int a, int b) { var builder = AsyncTaskMethodBuilder.Create(); var stateMachine = new Foo__0(); // initialize state machine fields stateMachine.a = a; stateMachine.b = b; stateMachine.state = -1; stateMachine.builder = builder; builder.Start(ref stateMachine); // ^ similar to: ThreadPool.QueueUserWorkItem(this.MoveNext); return builder.Task; } }
  • 93. WHAT YOU SEE WHAT YOU DON’T SEE static class Program { static async Task Foo(int a, int b) { Console.WriteLine("Foo", a); await Task.Yield(); Console.WriteLine("Foo", b); } } struct Foo__0 : IStateMachine { AsyncTaskMethodBuilder builder; int state; int a; int b; public void MoveNext() { switch (this.state) { case -1: Console.WriteLine("Foo", this.a); var awaiter = Taks.Yield().GetAwaiter(); this.state = 0; this.builder.AwaitUnsafeOnCompleted(ref awaiter, ref this); // ^ awaiter.OnComplete(() => // ThreadPool.QueueUserWorkItem(this.MoveNext)); return; case 0: Console.WriteLine("Foo", this.b); builder.SetResult(); return; } } } static class Program { static Task Foo(int a, int b) { var builder = AsyncTaskMethodBuilder.Create(); var stateMachine = new Foo__0(); // initialize state machine fields stateMachine.a = a; stateMachine.b = b; stateMachine.state = -1; stateMachine.builder = builder; builder.Start(ref stateMachine); // ^ similar to: ThreadPool.QueueUserWorkItem(this.MoveNext); return builder.Task; } }
  • 94. WHAT YOU SEE WHAT YOU DON’T SEE static class Program { static async Task Foo(int a, int b) { Console.WriteLine("Foo", a); await Task.Yield(); Console.WriteLine("Foo", b); } } struct Foo__0 : IStateMachine { AsyncTaskMethodBuilder builder; int state; int a; int b; public void MoveNext() { switch (this.state) { case -1: Console.WriteLine("Foo", this.a); var awaiter = Taks.Yield().GetAwaiter(); this.state = 0; this.builder.AwaitUnsafeOnCompleted(ref awaiter, ref this); // ^ awaiter.OnComplete(() => // ThreadPool.QueueUserWorkItem(this.MoveNext)); return; case 0: Console.WriteLine("Foo", this.b); builder.SetResult(); return; } } } static class Program { static Task Foo(int a, int b) { var builder = AsyncTaskMethodBuilder.Create(); var stateMachine = new Foo__0(); // initialize state machine fields stateMachine.a = a; stateMachine.b = b; stateMachine.state = -1; stateMachine.builder = builder; builder.Start(ref stateMachine); // ^ similar to: ThreadPool.QueueUserWorkItem(this.MoveNext); return builder.Task; } }
  • 95. WHAT YOU SEE WHAT YOU DON’T SEE static class Program { static async Task Foo(int a, int b) { Console.WriteLine("Foo", a); await Task.Yield(); Console.WriteLine("Foo", b); } } struct Foo__0 : IStateMachine { AsyncTaskMethodBuilder builder; int state; int a; int b; public void MoveNext() { switch (this.state) { case -1: Console.WriteLine("Foo", this.a); var awaiter = Taks.Yield().GetAwaiter(); this.state = 0; this.builder.AwaitUnsafeOnCompleted(ref awaiter, ref this); // ^ awaiter.OnComplete(() => // ThreadPool.QueueUserWorkItem(this.MoveNext)); return; case 0: Console.WriteLine("Foo", this.b); builder.SetResult(); return; } } } static class Program { static Task Foo(int a, int b) { var builder = AsyncTaskMethodBuilder.Create(); var stateMachine = new Foo__0(); // initialize state machine fields stateMachine.a = a; stateMachine.b = b; stateMachine.state = -1; stateMachine.builder = builder; builder.Start(ref stateMachine); // ^ similar to: ThreadPool.QueueUserWorkItem(this.MoveNext); return builder.Task; } }
  • 96. AWAITER PATTERN Custom awaitable objects public readonly struct PromiseAwaiter<T> : INotifyCompletion { private readonly Promise<T> promise; public PromiseAwaiter(Promise<T> promise) { this.promise = promise; } #region mandatory awaiter methods public bool IsCompleted => promise.IsCompleted; public T GetResult() => promise.Result; public void OnCompleted(Action continuation) => promise.RegisterContinuation(continuation); #endregion }
  • 97. ASYNC METHOD BUILDER Custom async methods public struct PromiseAsyncMethodBuilder<T> { private Promise<T>? promise; #region mandatory methods for async state machine builder public static PromiseAsyncMethodBuilder<T> Create() => default; public Promise<T> Task => promise ??= new Promise<T>(); public void SetException(Exception e) => Task.TrySetException(e); public void SetResult(T result) => Task.TrySetResult(result); public void AwaitOnCompleted<TAwaiter, TStateMachine>(ref TAwaiter awaiter, ref TStateMachine stateMachine) where TAwaiter : INotifyCompletion where TStateMachine : IAsyncStateMachine { } public void AwaitUnsafeOnCompleted<TAwaiter, TStateMachine>(ref TAwaiter awaiter, ref TStateMachine stateMachine) where TAwaiter : ICriticalNotifyCompletion where TStateMachine : IAsyncStateMachine { } public void Start<TStateMachine>(ref TStateMachine stateMachine) where TStateMachine : IAsyncStateMachine { } public void SetStateMachine(IAsyncStateMachine stateMachine) { } #endregion }
  • 100.  Custom AsyncMethodBuilder “docs”: https://github.com/dotnet/roslyn/blob/master/docs/features/task-types.md  Building custom (affine) thread pool: https://bartoszsypytkowski.com/thread-safety-with-affine-thread-pools/  How Go scheduler works: https://www.youtube.com/watch?v=-K11rY57K7k REFERENCES