Ad
More Related Content
What's hot (20)
Similar to Event-driven microservices (20)
![[CPT DevOps Meetup] Developing Modern Applications in the Cloud](https://cdn.slidesharecdn.com/ss_thumbnails/devopsdeveloping-modern-applications-in-the-cloud-no-notes-190718062311-thumbnail.jpg?width=560&fit=bounds)
Ad
More from Andrew Schofield (9)
Ad
Recently uploaded (20)
![Get & Download Wondershare Filmora Crack Latest [2025]](https://cdn.slidesharecdn.com/ss_thumbnails/revolutionizingresidentialwi-fi-250422112639-60fb726f-250429170801-59e1b240-thumbnail.jpg?width=560&fit=bounds)
![Download Wondershare Filmora Crack [2025] With Latest](https://cdn.slidesharecdn.com/ss_thumbnails/neo4j-howkgsareshapingthefutureofgenerativeaiatawssummitlondonapril2024-240426125209-2d9db05d-250419-250428115407-a04afffa-thumbnail.jpg?width=560&fit=bounds)
Event-driven microservices
- 1. Apache Kafka © 2019 IBM Corporation Event-Driven Microservices Using IBM Apache Kafka Andrew Schofield Chief Architect, Event Streams IBM Hursley Park Event-Driven Architecture meetup
- 2. © 2019 IBM Corporation What are microservices? Microservices is a technique for structuring an application as a collection of services • Self-contained with clear interfaces and a distinct purpose • Loosely coupled – communicate over a network • Independently deployable, scalable, maintainable and testable Microservices ApplicationMonolithic Application
- 3. © 2019 IBM Corporation Event-driven microservices Microservices communicate primarily with events, with APIs where required • Microservices can produce and consume events using publish/subscribe messaging • Events are handled by an event backbone • Data is eventually consistentMICROSERVICES APPLICATION API MICROSERVICE MICROSERVICE MICROSERVICE EVENT BACKBONE MICROSERVICE MICROSERVICE MICROSERVICE MICROSERVICE MICROSERVICE API SERVICE DISCOVERY PUBLISH PUBLISH SUBSCRIBE SUBSCRIBE
- 4. © 2019 IBM Corporation Comparing messaging patterns QUEUE 0 1 2 3 4 5 6 7 CONSUMER CONSUMER CONSUMER PRODUCER PRODUCER TOPIC 0 1 2 3 4 5 6 7 SUBSCRIPTION SUBSCRIPTION PRODUCER PRODUCER CONSUMER CONSUMER CONSUMER POINT-TO-POINT PUBLISH/SUBSCRIBE
- 5. © 2019 IBM Corporation Apache Kafka is an Open-Source Streaming Platform PUBLISH/SUBSCRIBE Read and write streams of events, like a traditional messaging system PROCESS Support scalable stream processing applications that react to events in real time STORE Store streams of data safely in a distributed, replicated, fault-tolerant cluster https://kafka.apache.org/ Kafka Cluster Producer ProducerProducer Consumer ConsumerConsumer Stream Processor Stream Processor Source Connector Sink Connector
- 6. © 2019 IBM Corporation Introducing the event backbone EVENT BACKBONE Microservic e Microservic e
- 7. © 2019 IBM Corporation Introducing the event backbone 1 Building Blocks 1 :: Event sources Microservic e Microservic e EVENT BACKBONE
- 8. © 2019 IBM Corporation Introducing the event backbone 1 2 Building Blocks 1 :: Event sources 2 :: Stream processing Microservic e Microservic e EVENT BACKBONE
- 9. © 2019 IBM Corporation Introducing the event backbone 1 2 App 3 Building Blocks 1 :: Event sources 2 :: Stream processing 3 :: Event archive Microservic e Microservic e EVENT BACKBONE
- 10. © 2019 IBM Corporation Introducing the event backbone App 1 2 3 4 Building Blocks 1 :: Event sources 2 :: Stream processing 3 :: Event archive 4 :: Notifications Microservic e Microservic e EVENT BACKBONE
- 11. © 2019 IBM Corporation A simple stream processing example EVENT BACKBONE Transactions
- 12. © 2019 IBM Corporation A simple stream processing example EVENT BACKBONE Transactions Alerts
- 13. © 2019 IBM Corporation A simple stream processing example EVENT BACKBONE Transactions Alerts SMS alerts Email alerts Push alerts
- 14. © 2019 IBM Corporation A simple stream processing example EVENT BACKBONE Send SMS Transactions Alerts Send emails Send push notifications SMS alerts Email alerts Push alerts
- 15. © 2019 IBM Corporation How to get started You could run an event storming workshop https://www.ibm.com/cloud/garage/architectures/eventDrivenArchitecture/event-storming- methodology Identify: Events Actors Data Commands
- 16. © 2019 IBM Corporation A more complex event-driven example https://ibm-cloud-architecture.github.io/refarch-kc/design/readme/
- 17. © 2019 IBM Corporation A more complex example – component view Fleet microservice Containers microservice Voyages microservice Orders microservice API API
- 18. © 2019 IBM Corporation A more complex example – component view Fleet microservice Containers microservice Voyages microservice Orders microservice API API
- 19. © 2019 IBM Corporation A more complex example – component view EVENT BACKBONE ships containers orders voyages Fleet microservice Containers microservice Voyages microservice Orders microservice API API
- 20. © 2019 IBM Corporation A more complex example – component view EVENT BACKBONE ships containers problems orders voyages Fleet microservice Containers microservice Voyages microservice Orders microservice Streaming analytics API API
- 21. © 2019 IBM Corporation Patterns for event-driven microservices You want: Loose coupling Data consistency Efficient queries You need PATTERNS, such as: Database per service Saga Event sourcing CQRS
- 22. © 2019 IBM Corporation Pattern – Database per service Each microservice persists its own data Protects independence of the microservice against external change Introduces complexity for data consistency across microservices EVENT BACKBONE ships containers problems orders voyages Fleet microservice Containers microservice Voyages microservice Orders microservice
- 23. © 2019 IBM Corporation Kafka log compaction for data replication 0 key:a val:A1 1 key:b val:B1 2 key:a val:A2 3 key:c val:C1 TOPIC PARTITION 4 key:c val:C2 PRODUCER Consumer takes latest values and builds own data store KEY VALUE a A2 b <deleted> c C2 SOURCE CHANGE LOG Read
- 24. © 2019 IBM Corporation Kafka log compaction for data replication 0 key:a val:A1 1 key:b val:B1 2 key:a val:A2 3 key:c val:C1 TOPIC PARTITION 4 key:c val:C2 KEY VALUE a A2 b <deleted> c C2 2 key:a val:A2 4 key:b val:C2 TOPIC PARTITION (rewritten) PERIODIC COMPACTION ELIMINATES DUPLICATE KEYS TO MINIMIZE STORAGE
- 25. © 2019 IBM Corporation Pattern – Sagas Orchestration of multi-step operations across microservices Data consistency across microservices without distributed transactions Programming can be complex, particularly for failure compensation EVENT BACKBONE Fleet microservice Containers microservice Voyages microservice Orders microservice 1 2 3 4 5
- 26. © 2019 IBM Corporation EVENT BACKBONE Pattern – Event sourcing Every state change to an object is captured as an event stored in sequence Replaying the events in order rebuilds the current state of the objects By subscribing to the events, you get an evolving copy of the objects’ state Query handler Command handler USER Order1 Created Order1 Updated Order2 Created Order1 Complete Order3 Created Order2 Canceled Order3 Updated Order3 Complete
- 27. © 2019 IBM Corporation Pattern – Event sourcing with Kafka Order2 Created Order2 Updated Order4 Created Order2 Complete Order4 Canceled Order3 Created Order3 Updated Order6 Created Order3 Complete Order3 Created Order6 Updated Order1 Created Order1 Updated Order5 Created Order1 Complete Order7 Created Order5 Canceled Order7 Updated Order7 Complete TOPIC orders PARTITION 0 TOPIC orders PARTITION 1 TOPIC orders PARTITION 2 CONSUMER GROUP A p0, offset 7 p1, offset 3 p2, offset 5 CONSUMER CONSUMER CONSUMER CONSUMER GROUP B p0, offset 7 p1, offset 3 p2, offset 5 CONSUMER CONSUMER
- 28. © 2019 IBM Corporation Pattern – Command Query Responsibility Segregation (CQRS) Atomic writes and efficient reads at the same time API split into commands (change state) and queries (read only) Complex in practice EVENT BACKBONE 1 2 3 4 5 Fleet MS WRITE MODEL Fleet MS READ MODEL Write-optimized store Read-optimized store Write API Read API
- 29. © 2019 IBM Corporation Summary Event-driven microservices offer an effective way to build loosely coupled applications Techniques such as event storming can be used to derive the objects, commands and events in an application Patterns such as event sourcing and sagas make the complexity manageable
- 30. © 2019 IBM Corporation Thank you Andrew Schofield Chief Architect, Event Streams IBM Hursley Park [email protected]
Editor's Notes
- #4: Why event-driven? Proper loose-coupling means asynchronous communication Enables responsive applications In the real world, things often take time to complete
- #6: Kafka is a great choice for the event backbone Publish/subscribe Stream history Partitioning – workload distribution and ordering
- #12: Stream processing takes a sequence of data (the stream) and applies a sequence of processing to each element in the stream. Optimised for this continuous, event-at-a-time processing Techniques such as pipelining, or batching of transactions.
- #16: Outcome is a design built from loosely coupled microservices linked through an event-driven architecture using one or more event backbones 6-8 people including domain experts and stakeholders – sticky notes, stand up, collaborate Domain event – past tense ”order completed” Actor – users Command – action/decision Data - needed for the commands Phases Domain events, placed on a timeline Commands Data Aggregates – group together related events and commands and data – potential boundary of microservices
- #22: Could use Kafka’s stream-table duality which layers a database on topic of a key-value-based stream
- #26: https://simplesource.io/simple_sagas_key_concepts.html
- #29: write subsystem to publish changes to an event log, and the query subsystem to materialize the views it requires by applying these changes