Scala usergroup stockholm - reactive integrations with akka streams
2 likes657 views
This document discusses Akka Streams, which provide asynchronous back pressured stream processing in Akka. It describes key Akka Stream concepts like sources, sinks, and flows. It also discusses how Akka Streams integrate with other technologies like Reactive Streams, Kafka, HTTP servers and clients. Alpakka is mentioned as a community for developing Akka Stream connectors.
![Akka Streams in 20 seconds:
val source: Source[Int, NotUsed] =
Source(0 to 200000)
val flow: Flow[Int, String, NotUsed] =
Flow[Int].map(_.toString)
val sink: Sink[String, Future[Done]] =
Sink.foreach(println)
val runnableGraph =
source.via(flow).to(sink)
runnableGraph.run()](https://image.slidesharecdn.com/scalausergroupstockholm-reactiveintegrationswithakkastreams-161019065425/85/Scala-usergroup-stockholm-reactive-integrations-with-akka-streams-16-320.jpg)
![Akka Streams in 20 seconds:
source.via(flow).to(sink)
Source[Int, NotUsed]
Flow[Int, String, NotUsed]
Sink[String, Future[Done]]](https://image.slidesharecdn.com/scalausergroupstockholm-reactiveintegrationswithakkastreams-161019065425/85/Scala-usergroup-stockholm-reactive-integrations-with-akka-streams-17-320.jpg)
![Streaming in Akka HTTP
http://doc.akka.io/docs/akka/2.4/scala/stream/stream-customize.html#graphstage-scala
“Framed entity streaming”
http://doc.akka.io/docs/akka/2.4/java/http/routing-dsl/source-streaming-support.html
HttpServer as a:
Flow[HttpRequest, HttpResponse]](https://image.slidesharecdn.com/scalausergroupstockholm-reactiveintegrationswithakkastreams-161019065425/85/Scala-usergroup-stockholm-reactive-integrations-with-akka-streams-33-320.jpg)
![Streaming in Akka HTTP
HttpServer as a:
Flow[HttpRequest, HttpResponse]
HTTP Entity as a:
Source[ByteString, _]
http://doc.akka.io/docs/akka/2.4/scala/stream/stream-customize.html#graphstage-scala
“Framed entity streaming”
http://doc.akka.io/docs/akka/2.4/java/http/routing-dsl/source-streaming-support.html](https://image.slidesharecdn.com/scalausergroupstockholm-reactiveintegrationswithakkastreams-161019065425/85/Scala-usergroup-stockholm-reactive-integrations-with-akka-streams-34-320.jpg)
![Streaming in Akka HTTP
HttpServer as a:
Flow[HttpRequest, HttpResponse]
HTTP Entity as a:
Source[ByteString, _]
Websocket connection as a:
Flow[ws.Message, ws.Message]
http://doc.akka.io/docs/akka/2.4/scala/stream/stream-customize.html#graphstage-scala
“Framed entity streaming”
http://doc.akka.io/docs/akka/2.4/java/http/routing-dsl/source-streaming-support.html](https://image.slidesharecdn.com/scalausergroupstockholm-reactiveintegrationswithakkastreams-161019065425/85/Scala-usergroup-stockholm-reactive-integrations-with-akka-streams-35-320.jpg)
Scala usergroup stockholm - reactive integrations with akka streams
- 1. akka streams Reactive Integrations with that just work™ Johan Andrén Scala Usergroup Stockholm 2016-10-18
- 2. Johan Andrén Akka Team Stockholm Scala User Group
- 3. Make building powerful concurrent & distributed applications simple. Akka is a toolkit and runtime for building highly concurrent, distributed, and resilient message-driven applications on the JVM
- 4. Actors – simple & high performance concurrency Cluster / Remoting – location transparency, resilience Cluster tools – and more prepackaged patterns Streams – back-pressured stream processing Persistence – Event Sourcing HTTP – complete, fully async and reactive HTTP Server Official Kafka, Cassandra, DynamoDB integrations, tons more in the community Complete Java & Scala APIs for all features What’s in the toolkit?
- 6. akka streams Asynchronous back pressured stream processing Source Sink Flow
- 7. akka streams Asynchronous back pressured stream processing Source Sink (possible) asynchronous boundaries Flow
- 8. akka streams Asynchronous back pressured stream processing Source Sink 10 msg/s 1 msg/s OutOfMemoryError!! Flow
- 9. akka streams Asynchronous back pressured stream processing Source Sink 10 msg/s 1 msg/s hand me 3 morehand me 3 more 1 msg/s Flow
- 10. akka streams Not only linear streams Source SinkFlow Source Sink Flow Flow
- 11. Reactive Streams Reactive Streams is an initiative to provide a standard for asynchronous stream processing with non-blocking back pressure. This encompasses efforts aimed at runtime environments (JVM and JavaScript) as well as network protocols http://www.reactive-streams.org
- 12. Part of JDK 9 java.util.concurrent.Flow
- 13. Reactive Streams RS Library A RS library B async boundary
- 14. Reactive Streams RS Library A RS library B async boundary Make building powerful concurrent & distributed applications simple.
- 15. The API Akka Streams Complete and awesome Java and Scala APIs (Just like everything in Akka)
- 16. Akka Streams in 20 seconds: val source: Source[Int, NotUsed] = Source(0 to 200000) val flow: Flow[Int, String, NotUsed] = Flow[Int].map(_.toString) val sink: Sink[String, Future[Done]] = Sink.foreach(println) val runnableGraph = source.via(flow).to(sink) runnableGraph.run()
- 17. Akka Streams in 20 seconds: source.via(flow).to(sink) Source[Int, NotUsed] Flow[Int, String, NotUsed] Sink[String, Future[Done]]
- 18. Materialization Gears from GeeCON.org,(it’s an awesome conf)
- 19. What is “materialization” really?
- 20. What is “materialization” really?
- 21. What is “materialization” really?
- 22. What is “materialization” really?
- 23. AlpakkaA community for Streams connectors http://blog.akka.io/integrations/2016/08/23/intro-alpakka
- 24. Alpakka – a community for Stream connectors Threading & Concurrency in Akka Streams Explained (part I) Mastering GraphStages (part I, Introduction) Akka Streams Integration, codename Alpakka A gentle introduction to building Sinks and Sources using GraphStage APIs (Mastering GraphStages, Part II) Writing Akka Streams Connectors for existing APIs Flow control at the boundary of Akka Streams and a data provider Akka Streams Kafka 0.11
- 25. Alpakka – a community for Stream connectors Existing examples: MQTT AMQP Streaming HTTP Streaming TCP Streaming FileIO Cassandra Queries “Reactive Kafka” (akka-stream-kafka) S3, SQS & other Amazon APIs Streaming JSON Streaming XML …
- 26. Alpakka – a community for Stream connectors Demo KafkaSink FileTailSource Kafka Bytes => Lines
- 27. Alpakka – a community for Stream connectors Demo Kafka parse line window last 10 Find hacker KafkaSource Sound the alarm!
- 28. Alpakka – a community for Stream connectors Kafka Stream Stream Stream Stream cluster
- 29. Akka Streams & HTTP streams & HTTP
- 30. Akka Streams / HTTP Quiz time!
- 31. Akka Streams / HTTP recv buffer send buffer 🚚 🚚 🚚 🚚 🚚 🚚 🚚
- 32. Akka Streams / HTTP recv buffer send buffer 🚚 🚚 🚚 🚚 🚚 🚚 🚚 🚚 🚚 🚚 🚑
- 33. Streaming in Akka HTTP http://doc.akka.io/docs/akka/2.4/scala/stream/stream-customize.html#graphstage-scala “Framed entity streaming” http://doc.akka.io/docs/akka/2.4/java/http/routing-dsl/source-streaming-support.html HttpServer as a: Flow[HttpRequest, HttpResponse]
- 34. Streaming in Akka HTTP HttpServer as a: Flow[HttpRequest, HttpResponse] HTTP Entity as a: Source[ByteString, _] http://doc.akka.io/docs/akka/2.4/scala/stream/stream-customize.html#graphstage-scala “Framed entity streaming” http://doc.akka.io/docs/akka/2.4/java/http/routing-dsl/source-streaming-support.html
- 35. Streaming in Akka HTTP HttpServer as a: Flow[HttpRequest, HttpResponse] HTTP Entity as a: Source[ByteString, _] Websocket connection as a: Flow[ws.Message, ws.Message] http://doc.akka.io/docs/akka/2.4/scala/stream/stream-customize.html#graphstage-scala “Framed entity streaming” http://doc.akka.io/docs/akka/2.4/java/http/routing-dsl/source-streaming-support.html
- 36. Streaming from Akka HTTP send buffer 🚚 🚚 OSApp EntrySource Json => Bytes Entry => Json TcpSink
- 37. Streaming from Akka HTTP send buffer 🚚 🚚 🚚 OSApp EntrySource Json => Bytes Entry => Json TcpSink 🚚 🚚
- 38. Streaming from Akka HTTP send buffer 🚚 🚚 🚚 OSApp EntrySource Json => Bytes Entry => Json TcpSink 🚚 🚚 No demand from TCP = No demand upstream = Source won’t read log => Bounded memory stream processing! Demo
- 39. Akka ❤ contributions Easy to contribute tickets: https://github.com/akka/akka/issues?q=is%3Aissue+is%3Aopen+label%3Aeasy-to-contribute https://github.com/akka/akka/issues?q=is%3Aissue+is%3Aopen+label%3A%22nice-to-have+%28low-prio%29%22 Akka Stream Contrib https://github.com/akka/akka-stream-contrib Mailing list: https://groups.google.com/group/akka-user Public chat rooms: http://gitter.im/akka/dev developing Akka http://gitter.im/akka/akka using Akka
- 40. Thanks for listening! @apnylle [email protected] Sample sources https://github.com/johanandren/akka-stream-samples/tree/scala-stockholm-2016-10