Understanding Akka Streams, Back Pressure, and Asynchronous Architectures
29 likes23,692 views
The term 'streams' has been getting pretty overloaded recently–it's hard to know where to best use different technologies with streams in the name. In this talk by noted hAkker Konrad Malawski, we'll disambiguate what streams are and what they aren't, taking a deeper look into Akka Streams (the implementation) and Reactive Streams (the standard). You'll be introduced to a number of real life scenarios where applying back-pressure helps to keep your systems fast and healthy at the same time. While the focus is mainly on the Akka Streams implementation, the general principles apply to any kind of asynchronous, message-driven architectures.
![Publisher[T] Subscriber[T]
Back-pressure explained](https://image.slidesharecdn.com/2016-understanding-akka-streams-async-lightbendwebinar-160623173705/85/Understanding-Akka-Streams-Back-Pressure-and-Asynchronous-Architectures-14-320.jpg)
![Akka Streams in 20 seconds:
// types:
Source[Out, Mat]
Flow[In, Out, Mat]
Sink[In, Mat]
// generally speaking, it's always:
val ready = Source(???).via(flow).map(_ * 2).to(sink)
val mat: Mat = ready.run()
// the usual example:
val f: Future[String] =
Source.single(1).map(_.toString).runWith(Sink.head)
Proper static typing!](https://image.slidesharecdn.com/2016-understanding-akka-streams-async-lightbendwebinar-160623173705/85/Understanding-Akka-Streams-Back-Pressure-and-Asynchronous-Architectures-39-320.jpg)
![Akka Streams in 20 seconds:
// types: _
Source[Int, Unit]
Flow[Int, String, Unit]
Sink[String, Future[String]]
Source.single(1).map(_.toString).runWith(Sink.head)](https://image.slidesharecdn.com/2016-understanding-akka-streams-async-lightbendwebinar-160623173705/85/Understanding-Akka-Streams-Back-Pressure-and-Asynchronous-Architectures-40-320.jpg)
![Akka Streams in 20 seconds:
// types: _
Source[Int, Unit]
Flow[Int, String, Unit]
Sink[String, Future[String]]
Source.single(1).map(_.toString).runWith(Sink.head)](https://image.slidesharecdn.com/2016-understanding-akka-streams-async-lightbendwebinar-160623173705/85/Understanding-Akka-Streams-Back-Pressure-and-Asynchronous-Architectures-41-320.jpg)
![Streaming in Akka HTTP
DEMO
http://doc.akka.io/docs/akka/2.4.7/scala/stream/stream-customize.html#graphstage-scala
“Framed entity streaming” https://github.com/akka/akka/pull/20778
HttpServer as a:
Flow[HttpRequest, HttpResponse]](https://image.slidesharecdn.com/2016-understanding-akka-streams-async-lightbendwebinar-160623173705/85/Understanding-Akka-Streams-Back-Pressure-and-Asynchronous-Architectures-49-320.jpg)
![Streaming in Akka HTTP
DEMO
http://doc.akka.io/docs/akka/2.4.7/scala/stream/stream-customize.html#graphstage-scala
“Framed entity streaming” https://github.com/akka/akka/pull/20778
HttpServer as a:
Flow[HttpRequest, HttpResponse]
HTTP Entity as a:
Source[ByteString, _]](https://image.slidesharecdn.com/2016-understanding-akka-streams-async-lightbendwebinar-160623173705/85/Understanding-Akka-Streams-Back-Pressure-and-Asynchronous-Architectures-50-320.jpg)
![Streaming in Akka HTTP
DEMO
http://doc.akka.io/docs/akka/2.4.7/scala/stream/stream-customize.html#graphstage-scala
“Framed entity streaming” https://github.com/akka/akka/pull/20778
HttpServer as a:
Flow[HttpRequest, HttpResponse]
HTTP Entity as a:
Source[ByteString, _]
Websocket connection as a:
Flow[ws.Message, ws.Message]](https://image.slidesharecdn.com/2016-understanding-akka-streams-async-lightbendwebinar-160623173705/85/Understanding-Akka-Streams-Back-Pressure-and-Asynchronous-Architectures-51-320.jpg)
Ad
More Related Content
What's hot (20)
Similar to Understanding Akka Streams, Back Pressure, and Asynchronous Architectures (20)
![[Tokyo Scala User Group] Akka Streams & Reactive Streams (0.7)](https://cdn.slidesharecdn.com/ss_thumbnails/2014-akka-streams-tokyo-140914201812-phpapp01-thumbnail.jpg?width=560&fit=bounds)
![Introduction to Akka Streams [Part-I]](https://cdn.slidesharecdn.com/ss_thumbnails/babystepsinakkastreampart-i-171117070802-thumbnail.jpg?width=560&fit=bounds)
Ad
More from Lightbend (20)
Ad
Recently uploaded (20)
Understanding Akka Streams, Back Pressure, and Asynchronous Architectures
- 1. WEBINAR Understanding Akka Streams, Back Pressure and Asynchronous Architectures by Konrad Malawski (@ktosopl)
- 2. Konrad `ktoso` Malawski Akka Team, Reactive Streams TCK, Persistence, HTTP
- 3. Konrad `@ktosopl` Malawski akka.io typesafe.com geecon.org Java.pl / KrakowScala.pl sckrk.com GDGKrakow.pl lambdakrk.pl
- 5. “Stream”
- 6. “Stream” What does it mean?!
- 7. Akka Streams Akka Streams && Reactive Streams
- 9. Why back-pressure? So you’ve built your app and it’s awesome.
- 10. Why back-pressure? Let’s not smash it horribly under load.
- 13. No no no…! Not THAT Back-pressure! No no no…! Not THAT Back-pressure! What is back-pressure?
- 15. Fast Publisher Slow Subscriber Push model
- 16. Subscriber usually has some kind of buffer. Push model
- 17. Push model
- 18. Push model
- 19. What if the buffer overflows? Push model
- 20. Use bounded buffer, drop messages + require re-sending Push model
- 21. Kernel does this! Routers do this! (TCP) Use bounded buffer, drop messages + require re-sending Push model
- 22. Increase buffer size… Well, while you have memory available! Push model
- 24. Reactive Streams explained Reactive Streams explained in 1 slide
- 25. Fast Publisher will send at-most 3 elements. This is pull-based-backpressure. Reactive Streams: “dynamic push/pull”
- 26. JEP-266 – soon…! public final class Flow { private Flow() {} // uninstantiable @FunctionalInterface public static interface Publisher<T> { public void subscribe(Subscriber<? super T> subscriber); } public static interface Subscriber<T> { public void onSubscribe(Subscription subscription); public void onNext(T item); public void onError(Throwable throwable); public void onComplete(); } public static interface Subscription { public void request(long n); public void cancel(); } public static interface Processor<T,R> extends Subscriber<T>, Publisher<R> { } }
- 27. Reactive Streams: goals 1) Avoiding unbounded buffering across async boundaries 2)Inter-op interfaces between various libraries
- 28. Reactive Streams: goals 1) Avoiding unbounded buffering across async boundaries 2)Inter-op interfaces between various libraries Argh, implementing a correct RS Publisher or Subscriber is so hard!
- 29. Reactive Streams: goals 1) Avoiding unbounded buffering across async boundaries 2)Inter-op interfaces between various libraries Argh, implementing a correct RS Publisher or Subscriber is so hard!
- 30. Reactive Streams: goals 1) Avoiding unbounded buffering across async boundaries 2)Inter-op interfaces between various libraries Argh, implementing a correct RS Publisher or Subscriber is so hard! You should be using Akka Streams abstractions instead!
- 31. Akka Streams Streams complement Actors, they do not replace them. Actors – distribution (location transparency) Streams – back-pressured + more rigid-blueprint
- 32. Akka is a Toolkit, pick the right tools for the job. Runar’s excellent talk @ Scala.World 2015 Asynchronous processing toolbox: Power Constraints
- 33. Akka is a Toolkit, pick the right tools for the job. Asynchronous processing toolbox: Constraints Power
- 34. Akka is a Toolkit, pick the right tools for the job. Single value, no streaming by definition. Local abstraction. Execution contexts. Power Constraints
- 35. Akka is a Toolkit, pick the right tools for the job. Mostly static processing layouts. Well typed and Back-pressured! Constraints Power
- 36. Akka is a Toolkit, pick the right tools for the job. Plain Actor’s younger brother, experimental. Location transparent, well typed. Technically unconstrained in actions performed Constraints Power
- 37. Akka is a Toolkit, pick the right tools for the job. Runar’s excellent talk @ Scala.World 2015 Location transparent. Various resilience mechanisms. (watching, persistent recovering, migration, pools) Untyped and unconstrained in actions performed. Constraints Power
- 39. Akka Streams in 20 seconds: // types: Source[Out, Mat] Flow[In, Out, Mat] Sink[In, Mat] // generally speaking, it's always: val ready = Source(???).via(flow).map(_ * 2).to(sink) val mat: Mat = ready.run() // the usual example: val f: Future[String] = Source.single(1).map(_.toString).runWith(Sink.head) Proper static typing!
- 40. Akka Streams in 20 seconds: // types: _ Source[Int, Unit] Flow[Int, String, Unit] Sink[String, Future[String]] Source.single(1).map(_.toString).runWith(Sink.head)
- 41. Akka Streams in 20 seconds: // types: _ Source[Int, Unit] Flow[Int, String, Unit] Sink[String, Future[String]] Source.single(1).map(_.toString).runWith(Sink.head)
- 42. Materialization Gears from GeeCON.org, did I mention it’s an awesome conf?
- 43. What is “materialization” really?
- 44. What is “materialization” really?
- 45. What is “materialization” really?
- 46. What is “materialization” really?
- 47. Akka Streams & HTTP streams & HTTP
- 48. Akka HTTP Joint effort of Spray and Akka teams. Complete HTTP Server/Client implementation. Learns from Spray’s 3-4 years history. Since the beginning with streaming as first class citizen. Side note: Lagom also utilises Akka Streams for streaming.
- 49. Streaming in Akka HTTP DEMO http://doc.akka.io/docs/akka/2.4.7/scala/stream/stream-customize.html#graphstage-scala “Framed entity streaming” https://github.com/akka/akka/pull/20778 HttpServer as a: Flow[HttpRequest, HttpResponse]
- 50. Streaming in Akka HTTP DEMO http://doc.akka.io/docs/akka/2.4.7/scala/stream/stream-customize.html#graphstage-scala “Framed entity streaming” https://github.com/akka/akka/pull/20778 HttpServer as a: Flow[HttpRequest, HttpResponse] HTTP Entity as a: Source[ByteString, _]
- 51. Streaming in Akka HTTP DEMO http://doc.akka.io/docs/akka/2.4.7/scala/stream/stream-customize.html#graphstage-scala “Framed entity streaming” https://github.com/akka/akka/pull/20778 HttpServer as a: Flow[HttpRequest, HttpResponse] HTTP Entity as a: Source[ByteString, _] Websocket connection as a: Flow[ws.Message, ws.Message]
- 52. Persistence Query (experimental) “The Query Side” of Akka Persistence
- 53. Persistence Query (experimental) Persistence Query Journals akka/akka-persistence-cassandra 0.16 akka/akka @ leveldb-journal 2.4.8 dnvriend/akka-persistence-jdbc 2.3.3 scullxbones/akka-persistence-mongo 1.2.5 …and more, that I likely forgot about. Implementation of “data-pump” pattern.
- 54. Akka + Kafka = BFF Reactive Kafka + Started by Krzysiek Ciesielski & Adam Warski @ SofwareMill.com
- 56. Kafka + Akka = BFF Akka is Arbitrary processing. Kafka is somewhat more than a message queue, but very focused on “the log”. Spark shines with it’s data-science focus.
- 57. Kafka + Akka = BFF
- 58. Kafka + Akka = BFF
- 59. Streams talking to Actors && Actors talking to Streams
- 60. Streams <=> Actors inter-op Source.actorRef (no back-pressure) Source.queue (safe) Sink.actorRef (no back-pressure) Sink.actorRefWithAck (safe)
- 62. Next steps for Akka Completely new Akka Remoting (goal: 1M+ msg/s (!)), (it is built using Akka Streams). More integrations for Akka Streams stages, also dynamic fan-in/out A.K.A.“the Hub”. Reactive Kafka polishing and stable release with SoftwareMill. “Confirmed Streams” work from Reactive Kafka generalised. Akka Typed likely to progress again. Of course, continued maintenance of Cluster and others.
- 63. Upgrade your grey matter Two free O’Reilly eBooks by Lightbend DOWNLOAD NOWDOWNLOAD NOW
- 64. lightbend.com/pov Reactive Roundtable World Tour by Lightbend lightbend.com/reactive-roundtable Proof of Value Service Accelerate Project Success
- 65. Q/A ktoso @ lightbend.com twitter: ktosopl github: ktoso team blog: letitcrash.com home: akka.io