![[WSO2Con Asia 2018] Building Reactive Applications Using Event-driven Archite...](https://cdn.slidesharecdn.com/ss_thumbnails/wso2conasia2018-buildingreactiveapplicationsusingevent-drivenarchitecture-180810094630-thumbnail.jpg?width=560&fit=bounds)
Ad
More Related Content
What's hot (20)
Similar to Event Driven Architectures - Phoenix Java Users Group 2013 (20)
Ad
More from clairvoyantllc (12)
Ad
Recently uploaded (20)
Event Driven Architectures - Phoenix Java Users Group 2013
- 1. Event Driven Architectures Avinash Ramineni
- 2. • What is Event Driven Architecture • How is EDA related to SOA • Event Capture/Generation • Event Processing • Event Delivery Guarantees • Complex Event Processing • Events, Internet of Things and Big Data Agenda
- 3. • Significant change in the state of an Object – Bank Transaction – Students grade gets posted – New Order gets created – Customer Profile gets updated – Change in Heart Beat • Characteristics of an Event – Events are broadcast – Communications are asynchronous – Events are fine grained Event
- 4. • Handling of the Events as they occur and the transfer of events among the systems in a loosely coupled way • Enables Situation Awareness and enables sense-and-respond behavior Event Driven Architecture (EDA)
- 5. • Characteristics of EDA – Producer/Consumers are not aware of each other – Little or no statefullness – Distributed – Loosely Coupled – Platform and Language Independent – Reliable – Fast and Highly Scalable Event Driven Architecture (EDA)
- 6. EDA extension of SOA Computing with event objects REST, URIs and HTTP Event Processing RPC-style SOA Event-driven SOA SOA Web Architecture Component software done right Web-oriented architecture (WOA) Non-SOA EDA and CEP Simple Web sites
- 7. • EDA similar to SOA facilitates Agility by using Modular Design and Separation of Concerns • EDA fits well into organizations that have some SOA infrastructure in place • EDA adds another dimension / Possibilities to SOA – Events are generally stateless and do not have much content • While processing the events that do not have much data in them, the consuming applications call the services exposed to get the actual data • Publish – Subscribe Model – Similar to Asynchronous Messaging Architectures EDA extension to SOA
- 8. • Application Generated – Application generates an event when a business transaction completes – Events generated by parsing the access logs/ server logs – Polling web pages – capturing the events from the services • Database Generated – Database Trigger generated Events • On insert/update/delete – During database replication How to generate Events
- 9. • The network is reliable • Latency is zero • Bandwidth is infinite • The network is secure • Topology does not change • Transport cost is zero • Network is homogeneous Fallacies of Distributed Systems
- 10. • Send an email when a customers profile gets updated Customer Profile – Assumptions: • There is an Application A that provides a updateProfileService • There is an Application B that provides a sendEmailService • All the Events are delivered with Once and Only Once SLA and the Events are processed Usecase: 1 A Event Channel DB Update DB Profile Updated Event B
- 11. • The overhead involved in capturing events must be as low as possible to avoid impairing the performance of a business transaction. • Business Transaction and the Event generation needs to be in the same transaction – 1,2 needs to be part of a same transaction • Both 1 and 2 should succeed or both should fail Considerations while generating Events A Event Channel DB 1. Update DB 2. Profile Updated Event B
- 12. • Successful Update of Profile -- Call a service to publish a Profile update Event • Can a database transaction and Web service call be made atomic? Usecase: 1 – Web service try{ 1. Update Profile on DB 2. Call web service to publish profile updated event } catch(Exception e) { rollback(); } commit; try{ 1. Update Profile on DB } catch(Exception e) { rollback(); } commit(); 2. Call web service to publish profile updated event • Can Compensating Transaction work ?
- 13. • Successful Update of Profile -- Put a message on to Message Broker Usecase: 1 – JMS A Event Channel DB 1. Update DB 2. Profile Updated Event (JMS message) B • Possible to make 1 and 2 into a single transaction but it requires – Distributed Transaction.
- 14. • Follows 2 Phase Commit protocol • Very Chatty protocol and does a lot of logging to be able to recover from any failure scenario. • Too much overhead to 99.9% of the cases to handle less than 0.1% of cases • Increases the chances for deadlocks on the database. • Lowers the overall performance of the system. • Bane of scalability. Grinds the entire system to halt by adding overhead to each and every transaction. Distributed/XA Transactions (1)
- 15. • Availability of the Systems goes down. • XA Configuration is complicated • Difficult to test. • Many People tend to believe that using JTA implementation of transaction manager will take care of a XA transactions • Many a time people end up using JTA Manager even while dealing with single resource. Distributed/XA Transactions (2)
- 16. • Successful Update of Profile -- Put a message on to Message Broker Usecase: 1 – Local Database A Event Channel DB 1. Update DB 3. Batch process to send events to Event Channel B • 1 and 2 are in same transaction. 3 can be retried multiple times till they succeed. 2. Store Event
- 17. • Successful Update of Profile -- Put a message on to Message Broker Usecase: 1 – Local JMS A Event Channel DB 1. Update DB 3. Publish to Event Channel(JMS message) B • Possible to make 1 and 2 into a single transaction without a Distributed Transaction as long as Queue is backed by a same database queue Send Event to Local Queue 2
- 18. • Successful Update of Profile -- Put a message on to Message Broker / Call a web service to publish an Event • Have a Reconciliation Process in place to verify that there is an event generated for every Business Transaction – Have a unique Id (may be stored in DB) along with update and use that Id as Correlation Id with the Events Archive – If it events don’t tally up, recreate those events • Use the DB triggers to generate events – Write to a different local table – Call a web service to send the event to event channel (have seen people do that) Usecase: 1 – Other Ways
- 19. • Event Delivery Guarantees – Reliability • At least Once – Duplicate events can be delivered • At most Once – Some events can be lost • Once and only Once – Each is delivered Exactly once – Order of Delivery • In Order Guaranteed • In Order not Guaranteed Event Delivery
- 20. • Processing an Event and associated Business Transaction needs to be in the same transaction • 3,2 needs to be part of a same transaction • Both 3 and 2 should succeed or both should fail Considerations while Processing Events Event Channel 3. Acknowledge Event Process success 1. Profile Updated Event B 2. Send Email
- 21. • On Profile Update Event -- Call a service to send Email and on success acknowledge process successful Usecase: 1 – Consumer (1) onMessage try { sendEmail(); jms.commit() } catch (Exception e) { jms.rollback() } • What if jms.commit() fails ??
- 22. • If Jms.commit() fails , message gets delivered again Usecase: 1 – Consumer (2) onMessage try { if I have not processed this message successfully before { do some stuff in the database / Idempotency Logic /JMSRelivered flag jdbc.commit; } jms.commit() } catch (Exception e) { jms.rollback() }
- 23. • Idempotent Operation – An idempotent operations means that the result of a successful performed request is independent of the number of times it is executed. • Design consumer to be an Idempotent Consumer – By Understanding the Business rules it can be achieved relatively easy • Good Practice to make all the write operations Idempotent !! • Idempotency logic can be built on Event Ids, Business Ids – orderId, customerId, etc based on the Business scenario Work Around for At-Least-Once
- 24. • In-order delivery of events – Limit the number of event publisher/consumer instance for these specific events to 1 • Brings up questions like if one of the event is having an issue – all the events of that event type can come to a stand still – AVOID the need for In-order Delivery of Events • By limiting the content in the Event , we can avoid a lot of cases where In-order delivery is required • Using Message Selectors to select and aggregate the events can solve a few of the cases What If In-order Delivery is required
- 25. • Student Information System (SIS) maintain a list of students in a course roster. The students can be added or dropped from the course roster. SIS sends an event to Learning Management System (LMS) when the roster gets updated. • Lets us assume that SIS is sending the course roster as part of the event. • Processing of this Event requires the Roster Change events to be delivered in order to maintain the data from getting corrupted. Usecase: 2
- 26. Usecase: 2 – Requires In-Order SIS Event Channel DB 2. Deliver Roster Change Event LMS LMS DB 1. Roster Change Event
- 27. • Remove the roster data from the event and just pass a roster Id • On Delivery of the Event , the LMS system would use the rosterId from the event, makes a service call to the source system (SIS) to get the latest Roster data and builds the classes in LMS • Even if the events are delivered out of order , the roster data on LMS will be always be the latest. • By making the processor on LMS idempotent, we can even avoid the database update Usecase: 2 - Solution
- 28. Usecase: 2 SIS Event Channel DB 2. Deliver Roster Change Event LMS LMS DB 1. Roster Change Event 3. Get the latest roster for the Id 4. Update the LMS database
- 29. • Make Event Publishers responsible for making sure that events are generated for every Business transaction and are published to the event channel • Make Event Consumers responsible for making sure that they react to the Stimuli from the event • Dashboards in place to make sure failed Events are detected and acted upon • Collects Stats to see if the events need to be made finer or coarser • Maintain Event Catalogs and self service capabilities to subscribe to events Best Practices
- 30. Complex Event Processing (CEP) Traditional BI finds "needles in haystacks." Event-driven CEP finds "needles" in continuously arriving streams of "hay." • CEP tap the events happening with in an organization and provide a situation awareness and enable better and faster decisions – Identify complicated events that are inferred from event pattern detection, event pattern interpretors and so on – Ex: Monitoring – capture events from ticket system, network load, performance and send out notification bring up new servers into the pool
- 31. • With Big Data hype and with the “capture-it-all“ approach and IoT , EDA and Event Driven programming will get a huge boost in the coming years. • Events and EDA due to their characteristics will be very well suited for Ubiquitous Computing • We might start seeing an advent of Complex Event Applications by being used by various gadgets – The concepts of EDA and Events will be a perfect Match for gadgets like Basis ,FIT Events ,Big Data and Internet of Things
Editor's Notes
- #7: As defined by Gartner SOA is classified into multiple styles RPC-Style , WOA-style , Event Driven SOA
- #18: Application generated events are generally Coarse grained. Database generated events are generally grained
- #23: kind of ID and version, you can often detect duplicates yourself and so not require to pay the performance cost of XA