Developing microservices with aggregates (SpringOne platform, #s1p)
24 likes55,928 views
The document presents a discussion on the synergy between Domain Driven Design (DDD) aggregates and microservices, emphasizing the importance of maintaining consistency across aggregates using event sourcing. It outlines challenges in traditional domain models, such as reliance on ACID transactions and transactional integrity, and proposes an event-driven architecture for managing order and customer data. Event sourcing is highlighted as a solution for ensuring data consistency in a microservices context while noting its benefits and drawbacks.
Developing microservices with aggregates (SpringOne platform, #s1p)
- 1. @crichardson Developing Microservices with Aggregates Chris Richardson Founder of Eventuate.io Founder of the original CloudFoundry.com Author of POJOs in Action @crichardson [email protected] http://eventuate.io
- 2. @crichardson Presentation goal Show how Domain Driven Design Aggregates and Microservices are a perfect match
- 4. @crichardson About Chris Consultant and trainer focusing on modern application architectures including microservices (http://www.chrisrichardson.net/)
- 5. @crichardson About Chris Founder of a startup that is creating a platform that makes it easier for developers to write transactional microservices (http://eventuate.io)
- 7. @crichardson Agenda The problem with Domain Models and microservices Overview of aggregates Maintaining consistency between aggregates Using event sourcing with Aggregates Example application
- 8. @crichardson The Microservice architecture tackles complexity through modularization
- 10. @crichardson Microservice architecture Browser Mobile Device Store Front UI API Gateway Catalog Service Review Service Order Service … Service Catalog Database Review Database Order Database … Database HTML REST REST Apply X-axis and Z-axis scaling to each service independently
- 12. @crichardson But there are challenges…
- 13. @crichardson Product service Customer serviceOrder service Domain model = tangled web of classes Order OrderLine Item quantity … Address street city … Customer Product name price ? ? creditLimit
- 14. @crichardson Reliance on ACID transactions to enforce invariants BEGIN TRANSACTION … SELECT ORDER_TOTAL FROM ORDERS WHERE CUSTOMER_ID = ? … SELECT CREDIT_LIMIT FROM CUSTOMERS WHERE CUSTOMER_ID = ? … INSERT INTO ORDERS … … COMMIT TRANSACTION Simple and ACID
- 15. @crichardson But it violates encapsulation… BEGIN TRANSACTION … SELECT ORDER_TOTAL FROM ORDERS WHERE CUSTOMER_ID = ? … SELECT CREDIT_LIMIT FROM CUSTOMERS WHERE CUSTOMER_ID = ? … INSERT INTO ORDERS … … COMMIT TRANSACTION Private to the Order Service Private to the Customer Service
- 16. @crichardson .. and requires 2PC BEGIN TRANSACTION … SELECT ORDER_TOTAL FROM ORDERS WHERE CUSTOMER_ID = ? … SELECT CREDIT_LIMIT FROM CUSTOMERS WHERE CUSTOMER_ID = ? … INSERT INTO ORDERS … … COMMIT TRANSACTION
- 17. @crichardson 2PC is not a viable option Guarantees consistency BUT 2PC is best avoided Not supported by many NoSQL databases etc. CAP theorem 2PC impacts availability ….
- 18. @crichardson Doesn’t fit with the NoSQL DB “transaction” model
- 19. @crichardson Agenda The problem with Domain Models and microservices Overview of aggregates Maintaining consistency between aggregates Using event sourcing with Aggregates Example application
- 21. @crichardson Domain Driven Design - building blocks Entity Value object Services Repositories Aggregates Adopted Ignored (at least by me)
- 22. About Aggregates Cluster of objects that can be treated as a unit Graph consisting of a root entity and one or more other entities and value objects Typically business entities are Aggregates, e.g. customer, Account, Order, Product, …. Order OrderLine Item quantity productId productName productPrice customerId Address street city …
- 23. Aggregate: rule #1 Reference other aggregate roots via identity (primary key) Order OrderLine Item quantity productId productName productPrice customerId Address street city …
- 24. @crichardson Foreign keys in a Domain Model?!?
- 25. @crichardson Domain model = collection of loosely connected aggregates Order OrderLine Item quantity … Address street city … Customer Product name price
- 26. @crichardson Product service Customer serviceOrder service Easily partition into microservices Order OrderLine Item quantity … Address street city … Customer Product name price
- 27. @crichardson Aggregate rule #2 Transaction = processing one command by one aggregate
- 28. @crichardson Product service Customer serviceOrder service Transaction scope = service Order OrderLine Item quantity … Address street city … Customer Product name price
- 30. @crichardson Aggregate granularity If an update must be atomic then it must be handled by a single aggregate Therefore Aggregate granularity is important
- 32. @crichardson Agenda The problem with Domain Models and microservices Overview of aggregates Maintaining consistency between aggregates Using event sourcing with Aggregates Example application
- 33. @crichardson Customer management How to maintain data consistency between aggregates? Order management Order Service placeOrder() Customer Service updateCreditLimit() Customer creditLimit ... has ordersbelongs toOrder total Invariant: sum(open order.total) <= customer.creditLimit ?
- 35. @crichardson Use event-driven, eventually consistent order processing Order Service Customer Service Order created Credit Reserved Credit Check Failed Create Order OR Customer creditLimit creditReservations ... Order state total … create() reserveCredit() approve()/reject()
- 36. @crichardson Order Management Order id : 4567 total: 343 state = CREATED Customer Management Customer creditLimit : 12000 creditReservations: {} Customer creditLimit : 12000 creditReservations: { 4567 -> 343} Order id : 4567 total: 343 state = APPROVED Eventually consistent credit checking Message Bus createOrder() Publishes: Subscribes to: Subscribes to: publishes: OrderCreatedEvent CreditReservedEvent OrderCreatedEvent CreditReservedEvent
- 37. @crichardson Complexity of compensating transactions ACID transactions can simply rollback BUT Developer must write application logic to “rollback” eventually consistent transactions For example: CreditCheckFailed => cancel Order Money transfer destination account closed => credit source account Careful design required!
- 38. @crichardson How atomically update database and publish an event Order Service Order Database Message Broker insert Order publish OrderCreatedEvent dual write problem ?
- 39. @crichardson Failure = inconsistent system Order Service Order Database Message Broker insert Order publish OrderCreatedEvent X
- 40. @crichardson 2PC is not an option
- 41. @crichardson Reliably publish events when state changes
- 42. @crichardson Agenda The problem with Domain Models and microservices Overview of aggregates Maintaining consistency between aggregates Using event sourcing with Aggregates Example application
- 43. @crichardson Event sourcing = event-centric persistence Application Database Event store update publish X
- 44. @crichardson Event sourcing For each domain object (i.e. DDD aggregate): Identify (state changing) domain events, e.g. use Event Storming Define Event classes For example, Order events: OrderCreated, OrderCancelled, OrderApproved, OrderRejected, OrderShipped
- 45. @crichardson Persists events NOT current state Order status …. 101 ACCEPTED Order table X
- 46. @crichardson Persists events NOT current state Event table Entity type Event id Entity id Event data Order 902101 …OrderApproved Order 903101 …OrderShipped Event type Order 901101 …OrderCreated
- 47. @crichardson Replay events to recreate state Order state OrderCreated(…) OrderAccepted(…) OrderShipped(…) Events Periodically snapshot to avoid loading all events
- 48. @crichardson The present is a fold over history currentState = foldl(applyEvent, initialState, events)
- 49. @crichardson Event Store Application architecture Order 123 Customer 456 OrderCreated OrderApproved … CustomerCreated CustomerCreditReserved … CreateOrder UpdateOrder GetOrder Subscribe Order Service CreateCustomer UpdateCustomer GetCustomer Subscribe Customer Service
- 50. @crichardson Request handling in an event sourced application HTTP Handler Event Store pastEvents = findEvents(entityId) Order new() applyEvents(pastEvents) newEvents = processCmd(someCmd) saveEvents(newEvents) (optimistic locking) Order Service applyEvents(newEvents)
- 51. @crichardson Event Store publishes events consumed by other services Event Store Event Subscriber subscribe(EventTypes) publish(event) publish(event) Customer update() Customer Service
- 52. @crichardson Event Store publishes events consumed by other services Event Store Event Subscriber subscribe(EventTypes) publish(event) publish(event) CQRS View update() Service Xyz send notifications …
- 53. Event store = database + message broker Hybrid database and message broker Implementations: Home grown/DIY geteventstore.com by Greg Young http://eventuate.io (mine) Event Store Save aggregate events Get aggregate events Subscribe to events
- 54. @crichardson Benefits of event sourcing Solves data consistency issues in a Microservice/NoSQL based architecture Reliable event publishing: publishes events needed by predictive analytics etc, user notifications,… Eliminates O/R mapping problem (mostly) Reifies state changes: Built in, reliable audit log temporal queries Preserved history More easily implement future requirements
- 55. @crichardson Drawbacks of event sourcing… Requires application rewrite Weird and unfamiliar style of programming Events = a historical record of your bad design decisions Must detect and ignore duplicate events Idempotent event handlers Track most recent event and ignore older ones …
- 56. @crichardson … Drawbacks of event sourcing Querying the event store can be challenging Some queries might be complex/inefficient, e.g. accounts with a balance > X Event store might only support lookup of events by entity id Must use Command Query Responsibility Segregation (CQRS) to handle queries application must handle eventually consistent data
- 57. @crichardson Agenda The problem with Domain Models and microservices Overview of aggregates Maintaining consistency between aggregates Using event sourcing with Aggregates Example application
- 58. @crichardson Orders and Customers example Kafka Rest API Event Store Event Store Adapter Customer Service Rest API Order Service Customer Aggregate Order Aggregate Rest API Order History Service MongoDB CQRS View Event Store Adapter Event Store Adapter
- 59. @crichardson Customer microservice Customer Service Customer Controller Spring Cloud Stream Kafka Customer Aggregate Rest API Customer Event Handlers Event Store Event Store Adapter
- 60. @crichardson The Customer aggregate Money creditLimit Map<OrderId, Money> creditReservations Customer List<Event> process(CreateCustomerCommand cmd) { … } List<Event> process(ReserveCreditCommand cmd) { … } … void apply(CustomerCreatedEvent anEvent) { … } void apply(CreditReservedEvent anEvent) { … } … State Behavior
- 61. @crichardson Familiar concepts restructured class Customer { public void reserveCredit( orderId : String, amount : Money) { // verify // update state this.xyz = … } public List<Event> process( ReserveCreditCommand cmd) { // verify … return singletonList( new CreditReservedEvent(…); ) } public void apply( CreditReservedEvent event) { // update state this.xyz = event.xyz }
- 65. @crichardson Creating a Customer save() concisely specifies: 1.Creates Customer aggregate 2.Processes command 3.Applies events 4.Persists events
- 66. @crichardson Event handling in Customers Durable subscription name 1. Load Customer aggregate 2. Processes command 3. Applies events 4. Persists events Triggers BeanPostProcessor Publish message to Spring Cloud Stream (Kafka) when Customer not found
- 67. @crichardson Summary Aggregates are the building blocks of microservices Use events to maintain consistency between aggregates Event sourcing is a good way to implement a event-driven architecture