Akka Streams and HTTP
60 likes41,694 views
Akka Streams & HTTP provides reactive, asynchronous, and non-blocking streams for processing data and HTTP requests and responses. It builds upon Akka IO and the Reactive Streams initiative to allow stream-based topologies to be declared and run for tasks like processing big data, serving clients simultaneously with limited resources, and building distributed applications that integrate with external systems over HTTP. Key features include stream sources, sinks, and transformations along with a routing DSL for building HTTP servers and clients on top of Akka IO and HTTP Core.
![The Meat
22
trait Publisher[T] {
def subscribe(sub: Subscriber[T]): Unit
}
trait Subscription {
def request(n: Long): Unit
def cancel(): Unit
}
trait Subscriber[T] {
def onSubscribe(s: Subscription): Unit
def onNext(e: T): Unit
def onError(t: Throwable): Unit
def onComplete(): Unit
}
Not user-level API
Intended as SPI for interop](https://image.slidesharecdn.com/akkastreamsandhttp-150207032135-conversion-gate02/85/Akka-Streams-and-HTTP-22-320.jpg)
![Declaring and Running a Stream
32
val upper = Source(Iterator from 0).take(10)
val lower = Source(1.second, 1.second, () => Tick)
val source = Source[(Int, Tick)]() { implicit b =>
val zip = Zip[Int, Tick]
val out = UndefinedSink[(Int, Tick)]
upper ~> zip.left ~> out
lower ~> zip.right
out
}
val flow = Flow[(Int, Tick)].map{ case (x, _) => s"tick $x" }
val sink = Sink.foreach(println)
val future = source.connect(flow).runWith(sink)](https://image.slidesharecdn.com/akkastreamsandhttp-150207032135-conversion-gate02/85/Akka-Streams-and-HTTP-32-320.jpg)
![Declaring and Running a Stream
33
val upper = Source(Iterator from 0).take(10)
val lower = Source(1.second, 1.second, () => Tick)
val source = Source[(Int, Tick)]() { implicit b =>
val zip = Zip[Int, Tick]
val out = UndefinedSink[(Int, Tick)]
upper ~> zip.left ~> out
lower ~> zip.right
out
}
val flow = Flow[(Int, Tick)].map{ case (x, _) => s"tick $x" }
val sink = Sink.foreach(println)
val future = source.connect(flow).runWith(sink)](https://image.slidesharecdn.com/akkastreamsandhttp-150207032135-conversion-gate02/85/Akka-Streams-and-HTTP-33-320.jpg)
![Declaring and Running a Stream
34
val upper = Source(Iterator from 0).take(10)
val lower = Source(1.second, 1.second, () => Tick)
val source = Source[(Int, Tick)]() { implicit b =>
val zip = Zip[Int, Tick]
val out = UndefinedSink[(Int, Tick)]
upper ~> zip.left ~> out
lower ~> zip.right
out
}
val flow = Flow[(Int, Tick)].map{ case (x, _) => s"tick $x" }
val sink = Sink.foreach(println)
val future = source.connect(flow).runWith(sink)](https://image.slidesharecdn.com/akkastreamsandhttp-150207032135-conversion-gate02/85/Akka-Streams-and-HTTP-34-320.jpg)
![Stream Sources
• org.reactivestreams.Publisher[T]
• org.reactivestreams.Subscriber[T]
• Iterator[T] / Iterable[T]
• Code block (function that produces Option[T])
• scala.concurrent.Future[T]
• TickSource
• ActorPublisher
• singleton / empty / failed
• … plus write your own (fully extensible)
36](https://image.slidesharecdn.com/akkastreamsandhttp-150207032135-conversion-gate02/85/Akka-Streams-and-HTTP-36-320.jpg)
![Stream Sinks
• org.reactivestreams.Publisher[T]
• org.reactivestreams.Subscriber[T]
• ActorSubscriber
• scala.concurrent.Future[T]
• blackhole / foreach / fold / onComplete
• … or create your own
37](https://image.slidesharecdn.com/akkastreamsandhttp-150207032135-conversion-gate02/85/Akka-Streams-and-HTTP-37-320.jpg)
![A Quick View of the HTTP Model
51
case class HttpRequest(
method: HttpMethod = HttpMethods.GET,
uri: Uri = Uri./,
headers: immutable.Seq[HttpHeader] = Nil,
entity: RequestEntity = HttpEntity.Empty,
protocol: HttpProtocol = HttpProtocols.`HTTP/1.1`
) extends HttpMessage
case class HttpResponse(
status: StatusCode = StatusCodes.OK,
headers: immutable.Seq[HttpHeader] = Nil,
entity: ResponseEntity = HttpEntity.Empty,
protocol: HttpProtocol = HttpProtocols.`HTTP/1.1`
) extends HttpMessage](https://image.slidesharecdn.com/akkastreamsandhttp-150207032135-conversion-gate02/85/Akka-Streams-and-HTTP-51-320.jpg)
![A Simple Server Example
54
import Directives._
val binding = Http().bind("localhost", 8080)
binding startHandlingWith {
path("order" / HexIntNumber) { id =>
get {
complete(s"Received GET for order $id")
} ~
put {
complete((actor ? Put(id)).mapTo[String])
}
}
}](https://image.slidesharecdn.com/akkastreamsandhttp-150207032135-conversion-gate02/85/Akka-Streams-and-HTTP-54-320.jpg)
Ad
More Related Content
What's hot (20)
Similar to Akka Streams and HTTP (20)
Ad
More from Roland Kuhn (10)
Ad
Recently uploaded (20)
Akka Streams and HTTP
- 1. Akka Streams & HTTP Dr. Roland Kuhn @rolandkuhn — Akka Tech Lead
- 2. Streams Occur Naturally • traditionally: • file input/output • network connections • more modern: • processing big data with finite memory • real-time data processing (CEP) • serving numerous clients simultaneously with bounded resources (IoT, streaming HTTP APIs) 2
- 3. 3 What is a Stream? • ephemeral, time-dependent sequence of elements • possibly unbounded in length • therefore focusing on transformations «You cannot step twice into the same stream. For as you are stepping in, other waters are ever flowing on to you.» — Heraclitus
- 5. Reactive means Message Flow 5
- 6. Reactive means Message Flow 6 37% discount with code kuhnco
- 7. Distributed vs. Static Typing • in general a very tough problem • independent entities evolving concurrently • promising progress on Project Gålbma • in many specific cases no type evolution necessary • letting uniform messages flow in safe conduits • making streams even more interesting 7
- 8. Possible Solutions • the Traditional way: blocking calls 8
- 9. Possible Solutions • the Push way: buffering and/or dropping (optimized for fast consumer) 9
- 10. Possible Solutions • the Pull way: poor performance (optimized for fast producer) 10
- 11. Possible Solutions • the Reactive way: non-blocking & non-dropping & bounded 11
- 12. Reactive Streams
- 13. Origin and motivation • all participants face the same basic problem • all are building tools for their community • a common solution benefits everybody • interoperability to make best use of efforts • propose to include in future JDK 13
- 14. Goals • minimal interfaces—essentials only • rigorous specification of semantics • TCK for verification of implementation • complete freedom for many idiomatic APIs • specification should be efficiently implementable 14
- 15. Reactive Streams • asynchronous & non-blocking • flow of data • flow of demand • minimal coordination and contention • message passing allows for distribution across • applications, nodes, CPUs, threads, actors 15
- 16. A Data Market using Supply & Demand • data elements flow downstream • demand flows upstream • data elements flow only when there is demand • data in flight is bounded by signaled demand • recipient is in control of maximal incoming data rate 16 Publisher Subscriber data demand
- 17. Reactive Push–Pull • “push”—when consumer is faster • “pull”—when producer is faster • switches automatically between these • batching demand allows batching data 17 Publisher Subscriber data demand
- 18. Explicit Demand: One-to-many 18 demand data Splitting the data means merging the demand
- 19. Explicit Demand: Many-to-one 19 Merging the data means splitting the demand
- 20. Back-Pressure is Contagious • C is slow • B must slow down • A must slow down 20 CA B
- 21. • TCP for example has it built-in Back-Pressure can be Propagated 21 CA B networkhosts
- 22. The Meat 22 trait Publisher[T] { def subscribe(sub: Subscriber[T]): Unit } trait Subscription { def request(n: Long): Unit def cancel(): Unit } trait Subscriber[T] { def onSubscribe(s: Subscription): Unit def onNext(e: T): Unit def onError(t: Throwable): Unit def onComplete(): Unit } Not user-level API Intended as SPI for interop
- 23. Akka Streams
- 24. Declaring a Stream Topology 24
- 25. Declaring a Stream Topology 25
- 26. Declaring a Stream Topology 26
- 27. Declaring a Stream Topology 27
- 28. Declaring a Stream Topology 28
- 29. Declaring a Stream Topology 29
- 30. Declaring a Stream Topology 30
- 31. API Design • goals: • no magic • supreme compositionality • exhaustive model of bounded stream processing • consequences: • immutable and reusable stream blueprints • explicit materialization step 31 http://doc.akka.io/docs/akka-stream-and-http-experimental/1.0-M2/stream-design.html
- 32. Declaring and Running a Stream 32 val upper = Source(Iterator from 0).take(10) val lower = Source(1.second, 1.second, () => Tick) val source = Source[(Int, Tick)]() { implicit b => val zip = Zip[Int, Tick] val out = UndefinedSink[(Int, Tick)] upper ~> zip.left ~> out lower ~> zip.right out } val flow = Flow[(Int, Tick)].map{ case (x, _) => s"tick $x" } val sink = Sink.foreach(println) val future = source.connect(flow).runWith(sink)
- 33. Declaring and Running a Stream 33 val upper = Source(Iterator from 0).take(10) val lower = Source(1.second, 1.second, () => Tick) val source = Source[(Int, Tick)]() { implicit b => val zip = Zip[Int, Tick] val out = UndefinedSink[(Int, Tick)] upper ~> zip.left ~> out lower ~> zip.right out } val flow = Flow[(Int, Tick)].map{ case (x, _) => s"tick $x" } val sink = Sink.foreach(println) val future = source.connect(flow).runWith(sink)
- 34. Declaring and Running a Stream 34 val upper = Source(Iterator from 0).take(10) val lower = Source(1.second, 1.second, () => Tick) val source = Source[(Int, Tick)]() { implicit b => val zip = Zip[Int, Tick] val out = UndefinedSink[(Int, Tick)] upper ~> zip.left ~> out lower ~> zip.right out } val flow = Flow[(Int, Tick)].map{ case (x, _) => s"tick $x" } val sink = Sink.foreach(println) val future = source.connect(flow).runWith(sink)
- 35. Materialization • Akka Streams separate the what from the how • declarative Source/Flow/Sink DSL to create blueprint • FlowMaterializer turns this into running Actors • this allows alternative materialization strategies • optimization • verification / validation • cluster deployment • only Akka Actors for now, but more to come! 35
- 36. Stream Sources • org.reactivestreams.Publisher[T] • org.reactivestreams.Subscriber[T] • Iterator[T] / Iterable[T] • Code block (function that produces Option[T]) • scala.concurrent.Future[T] • TickSource • ActorPublisher • singleton / empty / failed • … plus write your own (fully extensible) 36
- 37. Stream Sinks • org.reactivestreams.Publisher[T] • org.reactivestreams.Subscriber[T] • ActorSubscriber • scala.concurrent.Future[T] • blackhole / foreach / fold / onComplete • … or create your own 37
- 38. Linear Stream Transformations • Deterministic (like for collections) • map, filter, collect, grouped, drop, take, groupBy, … • Time-Based • takeWithin, dropWithin, groupedWithin, … • Rate-Detached • expand, conflate, buffer, … • asynchronous • mapAsync, mapAsyncUnordered, flatten, … 38
- 39. Nonlinear Stream Transformations • Fan-In • merge, concat, zip, … • Fan-Out • broadcast, route, balance, unzip, … 39
- 40. Akka HTTP
- 41. Why do we add an HTTP module? • Akka is about building distributed applications • distribution implies integration • between internal (sub)systems ➜ akka-clusterbasedonakka-remote • with external systems ➜ akka-http(linguafrancaoftheinternet) 41
- 42. Akka HTTP—The Origins: Spray.IO • Fully embeddable HTTP stack based on Actors • focused on HTTP integration (toolkit) • server- and client-side • seamlessly integrated with Akka 42
- 43. Spray.IO Features • immutable, case class-based HTTP model • fast, lightweight HTTP client and server • powerful DSL for server-side API definition • fully async & non-blocking, actor-friendly, modular, testable • based entirely on Scala & Akka Actors 43
- 44. Spray.IO Weaknesses • handling of chunked requests is clunky, incomplete • dealing with large message entities can be difficult • some unintuitive corner-cases in routing DSL • deep implicit argument chains in some cases hard to debug • missing features, foremost websocket support 44
- 46. Akka HTTP is Spray 2.0 • addressing the weaknesses, polishing the features • Java API • simplified module structure • core improvement: fully stream-based 46
- 48. The Application Stack 48 O/S-level network stack Java NIO (JDK) Akka IO Akka HTTP Core Akka HTTP user- level
- 49. Akka IO • bridges Java NIO with Akka Actors or streams • supports TCP, UDP and SSL 49
- 50. Akka HTTP Core • implements HTTP/1.1 according to the RFCs • based upon the TCP facilities of Akka IO • exposed as freely reusable stream transformations • low-level and extensible implementation of HTTP model and spec 50
- 51. A Quick View of the HTTP Model 51 case class HttpRequest( method: HttpMethod = HttpMethods.GET, uri: Uri = Uri./, headers: immutable.Seq[HttpHeader] = Nil, entity: RequestEntity = HttpEntity.Empty, protocol: HttpProtocol = HttpProtocols.`HTTP/1.1` ) extends HttpMessage case class HttpResponse( status: StatusCode = StatusCodes.OK, headers: immutable.Seq[HttpHeader] = Nil, entity: ResponseEntity = HttpEntity.Empty, protocol: HttpProtocol = HttpProtocols.`HTTP/1.1` ) extends HttpMessage
- 52. Akka HTTP • builds upon Akka HTTP Core • adds (un)marshaling for HttpEntities • adds (de)compression (gzip / deflate) • high-level server-side routing DSL • type-safe and flexible • highly compositional building blocks (Directives) • includes common behaviors pre-canned 52
- 53. Stream Pipelines 53 TCP SSL HTTP App Requests Responses optional (not yet)
- 54. A Simple Server Example 54 import Directives._ val binding = Http().bind("localhost", 8080) binding startHandlingWith { path("order" / HexIntNumber) { id => get { complete(s"Received GET for order $id") } ~ put { complete((actor ? Put(id)).mapTo[String]) } } }
- 55. Summary
- 56. When can we have it? • currently pre-release versions: • reactive-streams 1.0.0-RC1 • Akka Streams & HTTP 1.0-M2 • still missing: • Akka HTTP client features • SSL integration • websockets • Akka Streams & HTTP 1.0 expected end of Feb’15 56
- 57. Resources • https://github.com/akka/akka/tree/release-2.3-dev • http://doc.akka.io/docs/akka-stream-and-http- experimental/1.0-M2/ • akka-user mailing list 57
- 58. ©Typesafe 2014 – All Rights Reserved