EDA Visuals
EDA Visuals
architectures
David Boyne
v0.0.3
2
EDA Visuals 3
David Boyne
Speaking at Events
If you are interested in having me speak at your upcoming events, I would be happy to
discuss the possibility. Whether it’s a conference, podcast, seminar, workshop, please feel free to
contact me.
Thanks for downloading this content, I hope it can help.
Table of Visuals
Choreography vs orchestration 6
Commands vs Events 8
Event Types 13
EventStorming 17
Events as Data 19
Message Delivery 27
Schema Management 32
Ubiquitous Language 37
Summary 40
Choreography vs orchestration
Choreography Using events to talk between services (also bounded contexts). Services interact
independently. Event-driven approach.
Example of services that can help with choreography are Amazon SQS, Amazon SNS or Amazon
EventBridge.
Orchestration Flow of state that is normally controlled by the orchestrator. Think of flow chart
or state machines. Might consider using orchestration within a bounded context. Normally follows
request/response.
Example would be AWS Step Functions.
Extra Resources
• Choreography vs Orhestration in the land of Serverless - Great blog on choreography and
orchestration through a serverless lense.
Sometimes you may want to store information first, and then send message downstream to consumers.
This allows you to offload data or large events into a database and then send lighter events
downstream.
An example use-case of this pattern would be to send large payloads (exceeding your event broker
limits) to downstream consumers. First you store, then you use the key downstream to get the
information back from the database.
Extra resources
• Claim Check Enterprise Integration pattern - Enterprise integration pattern for claim check, if
you want to know more head here.
• Publish large EventBridge events with the claim check pattern - Blog post that I wrote, to
help you use the claim check pattern with Amazon EventBridge.
• Code example using S3 and EventBridge - Pattern to show you how to use S3 and EventBridge
with claim check
Commands vs Events
Understanding the difference between commands and events can be important when building
event-driven architectures.
Sometimes downstream consumers need more information (sometimes seen when events are notifica-
tion events). Downstream consumers call external APIS or back to the producer to get information
they require. Be careful with this pattern as it could lead to coupling.
With content enricher pattern you can add an enricher in the middle to pick up messages/events
and enrich them before sending downstream to consumers.
Extra resources
• Content Enricher Enterprise Integration Pattern - Great book with tons of patterns and
information. Content online for free, recommend reading this.
• Enrich EventBridge events with Lambda - Blog post I wrote about enrichment pattern and
using Lambda to enrich your events
• Enrichment pattern with EventBridge Pipe - Pattern that I wrote for ServerlessLand that uses
EventBridge Pipes to enrich data before sending to downstream consumers.
When you dive into event-driven architecture you will see one of the main benefits is “loosely
coupled” services. In fact many people will say “producers should not know about consumers”, this
is technically true. . . .. but overlooked.
Technically our producers should not know about consumers but operationally as humans we do.
Common questions come up when building EDA applications over time:
1. What events/messages is this service producing?
2. What events/messages can I consume from this service?
3. What is the format of these events/messages?
4. What schema version do I use?
5. Who is producing what?
6. Who is consuming what?
These are common questions that will arise when building EDA applications.
Here are three areas that I consider that can help:
Using README files Simple and cheap. If you want to document your schema, or what your
service is publishing or subscribing too, maybe readme files can help? Anything is better than
nothing. (maybe you think nothing is OK, but as you scale your app high chance you might need
some form of discovery)
AsyncAPI AsyncAPI is an open source specification for defining asynchronous APIS. The
community has been growing over the past few years and many large organistions using it to help
them define and write standards for producers and consumers. Community has a wide range of
tools to help with integrating and documentation too.
EventCatalog I’m of course biased here, this is my own open source project called EventCatalog.
I designed this to help people document their EDA applications, powered by markdown files and
custom plugins. You can connect any system you want to EventCatalog and generate markdown
files. This web interface gives your team a new visual way to navigate producers, consumers and
domains. See example in action here.
Extra resources
• AsyncAPI - Open-Source tools to easily build and maintain your event-driven architecture.
All powered by the AsyncAPI specification, the industry standard for defining asynchronous
APIs.
• EventCatalog - EventCatalog is an Open Source project that helps you document your events,
services and domains.
Event Types
When building EDA applications, it’s important to know the different types of events you can
publish, each has their own trade-offs.
Notification Events
• Minimal information
• Less risk of data being out of sync
• Consumers often need to fetch data
• Increases architecture coupling (callback for info)
• Producer/Consumer contracts kept minimum
• Lower risk of breaking contracts
Delta events
• Stores difference between old/new
• Examples seen in change data capture events
• Can reduce complex in consumers needing to figure out what has changed
Domain Events
• Events that raised in the same bounded context
• Some folks reference these as internal events raised in bounded context
• Some folks reference these as business “important” events.
• People also refer to “integration” events. These are events used for integrations.
Extra Resources
• Best practices to design your events in event-driven applications - A talk I gave in 2022 around
event design and trade-offs to consider.
• What do you mean by “Event Driven?” - Martin Fowler dives into event-driven architecture
and talks about the different types of events
• The Event-Carried State Transfer Pattern - A great blog post on what ESCT is
• The event notification pattern - A great blog post on what notification events are
Event identification and design within EDA applications is important, the more I researched
and learnt about event design the more I believe it’s a core part of an EDA application.
When we start our journey building EDA applications, we raise events from producers to downstream
consumers, but what goes into our events, the structure, naming conventions, or the ability to
identify our events is often overlooked.
“Event first thinking” are just some of my thoughts around all this and topics I presented last year
at re:invent 2022 and EDA day in London 2022.
1. Identify
• Recommended using Event Storming with domain experts in your org to highlight the behaviour
of your system and events that flow through.
• Find your bounded context and domains within your systems.
• Define a Ubiquitous language with your org to help communication between teams/parties.
2. Define
• Think about what you want in your events, do you want them to be notification, delta or
event-carried state transfer or maybe something else?
• Think about documentation for your event, producers and consumers. How will people find
the events you are publishing, what are the contracts? I created EventCatalog that might be
able to help.
• Define naming conventions for your events, use these across your org and set standards, it can
help.
3. Implement
• When you implement events in your producers think about shared libraries you might want
to write. If you have standards, naming conventions or metadata for example, use shared
libraries in your org to help. Can save time.
• Understand the design trade-offs when you choose different event types, understand them then
implement them.
• Understand events “internal and external”. Are your events using within the same bounded
context, are they used to communicate between bounded context? Depending on what kind of
events they are will determine the impact of contracts and breaking changes.
Extra resources
• Best practices to design your events in event-driven applications - A talk I gave in 2022 around
event design and trade-offs to consider.
EventStorming
Figure 8: EventStorming
Need to figure out what events your system has? Or what commands generate events? Need to
identify bounded context within your system? EventStorming can help.
EventStorming is a great method to help you identify your events, commands, aggregates and
domains. EventStorming can also be used to help identify areas of duplication and complexity
within your system.
It’s important we understand the behaviour of the systems we try and build, event storming brings
domain experts and engineers together to get a shared understanding and start to identify the
behaviour of the system.
Extra resources
• What is Event Storming? - Wiki link to that describes what event storming is
• EventStorming.com - Website with many resources, workshops and book about EventStorming.
• Awesome EventStorming - Huge list of resources to learn more.
Events as Data
Using Event Sourcing to capture events in your system. The idea of capturing a set of events and
using streams like “Change Data Capture” to process the information downstream.
Other systems have their own derived data using the events and create their own views.
When designing your events, I believe it’s important to have explicit events for your event-driven-
architectures. Define your events, define schemas and make sure the event itself is clear and intent
can be understood. Also note producers that do not provide contracts (schemas) have increased
chance of producing implicit events for downstream consumers.
Explicit Events
• Be clear on the event and it’s intent
• Use clear naming conventions
• Use schemas to help define contracts
• Gives stability to both producers and consumers
Implicit Events
• Unclear event intent, try to avoid
• Unclear naming conventions
• Assumptions can be made
• Breaking changes possible without contract/schema.
Extra Resources
• EventStorming - EventStorming can help you identify your events in your system, using other
stake holders it can be a natural way of defining explicit events, as events naming is often
discussed.
• EDA Documentation - Documentation can help have explicit events. If consumers can discover
and understand your events you are half way there.
• Event First Thinking - Events are important, designing them, thinking about them, taking
your time. Treat events as first-class citizens with Event First Thinking.
• Event Types - Understand event types can help you along your event design path.
• Commands vs Events - Two different things. Be clear if your message is a command or event.
• Building Event-Driven Microservices - Thoughts generated from notes from this book.
Good parts
• Flexibility - The ability for consumers to come and go without impacting the producer. When
new business requirements occur, existing events can be listening too and consumers can be
created.
• Process in real time - Some systems can allow for events to be processed in real-time or
near-real time. Thinking about the user experience, could this lead to a better UX?
• Fault tolerance - Consumers are decoupled by nature, if a consumer was to break or fail to
process the event, the failure is isolated to that consumer.
• Idempotency - Key to think about idempotency. If the same event was to be replayed into
your consumer, what will happen? Avoid side effects, have the same outcome if event replays
were to happen.
When you build event-driven applications you can use events to communicate between services and
boundaries. Many people use EventStorming to highlight events and business domains.
Within a bounded context you can have many different services and communicate between these
services using events, sometimes these are referred to as “private”, “internal” or “domain”
events (depending on what you read. . . ). These events belong within the bounded context, they
can be exposed to implementation details and raise events assuming that downstream consumers
understand the domain language used in the boundary and implementation details, they are “private”
and not meant for “public” consumption.
Using events to communicate between between bounded contexts (remember this can be within
in your organization or outside your organization) can be referred to as “public” or “integration”
events. Event contracts are important here, and ideally you do not want to expose private or
implementation details of your domain in these. Consider a public language to communicate between
systems (Defining a ubiquitous Language can help here).
Private/Internal Events
• Can expose internal implementation details
• Use a language that the domain understands, external domains may not understand this
• Contract is important, but depending on how “close” the services are within boundary, could
be relaxed?
Public/Integration Events
• Should not really expose implementation details of the domain
• Events use a language that is shared between the business or boundaries, no assumptions
made
• Contract is extremely important. Outside your domain you have less knowledge of who is
consuming your events, don’t break them.
Extra resources
• Learning Domain Driven Design - Some notes taken from the book “Learning Domain Driven
Design” by Vladik Khononov.
Software changes all the time. Requirements change, technology changes and patterns change. Using
event-driven architectures can be great, as they allow you to be agile and adapt to change fast, but
the road to implementing event-driven architecture does not happen overnight.
Implementing an event-driven architectures is a journey, a technical journey but also a
journey for your organisation. There are many benefits to building event-driven architectures but
the implementation itself is a continous effort (like all software we write!).
If you have legacy code and you want to move to an EDA application, you can use migration
strategies to slowly migrate services into an EDA landscape (if it makes sense)! If you are looking
to migrate I recommend using EventStorming to identify the behaviour of the system and start
raising some basic events and consumers. Grow the producers and consumers over time and start to
decouple your applications. Find the natural bounded context in your business domain and start to
communicate with events.
If you want to build event-driven applications or already do, I believe it’s a journey and it will be a
continuous journey, paired with ever changing business requirements EDA can be a great option to
remain agile and adapt to change.
Start small, and grow your implementation over time.
Extra resources
• EventStorming Visual - Visual to help you learn about EventStorming.
• EDA Guide - Short guide to help you get started with event driven architectures.
Message Delivery
Different systems will provide different messaging solutions, understanding these can help you
understand how your messages will be given to downstream services.
At-most-once delivery
• Delivery means message will be delivered once
• If it fails it can be lost
At-least-once delivery
• Message may duplicate to consumer.
• Multiple attempts can be made to deliver message.
• Important to have idempotent consumers
• Multiple attempts can be made to deliver the message to the target until one succeeds. This
means messages can be duplicated but not lost.
Exactly-once delivery
• Message is given to the target exactly once
Extra resources
• At-least-once delivery - Short blog post to help.
• Exactly-once delivery - Short blog post to help.
Message Queue Messages are put onto a queue and a consumer consumes the message and
processes them. Messages are acknowledged as consumed and deleted afterwards. Messages are split
between consumers which makes it hard to communicate system with events.
Example of this would be Amazon SQS. Publish messages to the queue and then listen to them,
process them and they are removed from the queue.
Event Broker Event brokers are a push system, they push these events downstream to consumers.
Example of this would be Amazon EventBridge.
Extra Resources
• Building Event Driven Microservices - Notes mainly from this book by Adam Bellemare
When sending data between bounded context you have a few options to handle the data/contracts.
Conformist Pattern
• System A sends data to System B. System B does no translation of the data before using it
within it’s bounded context.
• System B conforms to the data/contract.
Anti-Corruption Layer
• System B does not want to conform to the contract of System A.
• System B translates the data into a domain model it can understand.
Open-Host Service
• System A sends data to System B but translates it before it is sent.
• Public language is used between systems.
• Integration details of System A is still locked away, but public inteface/message exposed.
Extra Resources
• Learning Domain Driven Design - Notes taken from the book “Learning Domain Driven
Design” by Vladik Khononov.
When building event-driven applications it’s important to know the responsibility of the consumer
and producer. This diagram represents my thoughts on it, but I think from design to design it may
differ.
Use this diagram as a base, but you may want to adapt it based on your architecture.
Schema Management
As EDA applciations grow developers often find schemas to be a more important part of their EDA
application.
1. What events can I listen to?
2. How are we going to manage change in our events?
3. What do the events look like?
4. What goes into our events?
Extra Resources
• AsyncAPI - AsyncAPI specification, the industry standard for defining asynchronous APIs.
• CloudEvents - A specification for describing event data in a common way
• Event Payload Standards - Blog post about Amazon EventBridge payload standards, lessons
can be applied anywhere.
When we design our architecture and build our services it can be useful to know the different types
of communication patterns. Although pretty simple, I think it’s important to know the difference
between sync and async.
Sync
• Normally seen as the “request/response” pattern. Example would be taking a phone call, you
need both parties there.
• An example of this could be a command, many API calls want something to happen.
• Fails fast, simple and low latency
• What happens when many requests come in? Need to scale the service, could you use Async
pattern instead? Something to consider.
Async
• With EDA a producer can fire an event and forget it and allow downstream consumers to
process it (if they want. . . )
• With messages, a message could be added to queue and downstream services control the speed
of injestion and process async.
• Async provides a loose coupling between services/systems.
• Receiver can control the rate of consumption of events/messages
• Example of this would be Amazon SQS.
Extra resources
• AWS: Re:invent 2022 - Building next-gen applications with event-driven architectures - Video
on building EDA applications and speaker talks through communication patterns.
When you start building event-driven applications you start to write producers and consumers and
you start to see value quite fast. Over time as more producers and consumers are added you will
start to face some common issues (these issues listed are all problems I have experienced and
seen in the community)
Lack of idempotency handling Consumers may process your events more than once (e.g. if
events need to replay, or failures etc), your consumers need to handle this. You need to make sure
your consumers produce the same outcome regardless of how many times it was called with the
same event, having side effects in your consumers (different results) can lead to issues.
Mistaking commands for events An easy one when you get started, mistaking commands vs
events. Here is a visual to help you understand the difference.
Multi purpose events When you start designing and implementing your events, it’s easy just to
add on that one extra field. . . . “what if we add a type field here”, this works and might provide
value, but over time your consumers might have a hard time understanding the intent of the event.
Rather than having one event to rule them all, why not split them out? Something to think about. . . .
Lack of documentation Let’s face it, many people don’t like to write documentation, especially
as you start to build your EDA applications. Overtime as you add producers and consumers it can
be hard to keep track of who is producing what, and who is consuming what. Adding documentation
can help. Here is visual with resources to help you.
Leaking implementation details When you have clear boundaries of services, it can be easy
to leak implementation details of your service in your event. Make your events explicit, don’t get
consumers to guess what is happening, don’t confuse them with implementation details.
Lack of standards/structure Overtime you might have 10s or 100s of events in your architecture,
think about what goes into these events, maybe you might use CloudEvents or define your own
standards, spend time here and think about it.
Extra resources
• Document your event-driven architectures - Visual here to help you understand ways to
document your event-driven architectures.
• Identify domains and events with EventStorming - Think about how you are going to identify
events in your system. EventStorming can help.
• Message delivery - How are your messages/events getting to your downstream consumers?
Understanding this can help.
• Events are important - Event First Thinking - Treat events as a first class citizen in your
architecture, doing this from the start can help you.
• Know the difference of types of events you raise - Notification events, Delta events, Event
carried state transfer. . . what does it mean? Important to know.
• Commands vs Events - Quickly learn the difference between commands and events, 5min here
can save pain later on.
• Passing events through context - Find your business domains and use events to communicate,
you might need to transform them before consumers can understand them. . . that’s OK, look
at these patterns.
Ubiquitous Language
It can be time consuming or even frustrating when teams within an organisation are using terms
to mean different things depending on the domain. For example a “policy” within one team
could mean something completely different within another. . . . This leads to mental translations of
models/business terms and can lead to confusion and assumptions.
Having a “Ubiquitous Language” means parties that communicate between each other speak in the
same language. Define standards for terms, share them with each other and use them to describe
business domain.
This is not directly associated to “event-driven architectures” but domain driven design plays a huge
part in EDA. Having a shared language can help you name events, boundaries and build your EDA
applications.
Extra Resources
• Best practices to design your events in event-driven applications - Blog post by Martin Fowler
on Ubiquitous Language.
• Domain Driven Design Book - Domain-Driven-Design book, where Eric Evans talks about
Ubiquitous Language.
• Learning Domain Driven Design - Notes in this design taken from the book “Learning Domain
Driven Design” by Vladik Khononov.
EDA Guide on Serverlessland.com Serverlessland.com hosts over 400 AWS Serverless patterns,
60 Step Functions workflow patterns and over 40 serverless code snippets, we also have a new EDA
section to help the community learn more about building event-driven architectures. If you want to
know more you can visit https://serverlessland.com/event-driven-architecture/intro
30 Days of Serverless and EDA For those who want to get hands on building serverless event-
driven-architectures on AWS, we have created a new program for 30 days of Serverless. These are a
collection of videos we have put together to help you learn serverless and event-driven-architectures
from the ground up. If you want to learn more you can visit https://serverlessland.com/reinvent2022/
30dayssls
Summary
EDA visuals are small bite sized visuals about event-driven architectures. You can use the visuals
to help you get a high level overview of areas of event-driven architectures, and use the resources to
dive deeper.
The visuals in this document are from https://serverlessland.com/event-driven-architecture/visuals
and this document will be generated again with every new visual added.
Hopefully you find this content useful, and feel free to connect with me if you want to learn more.
• Twitter: https://twitter.com/boyney123
• LinkedIn: https://www.linkedin.com/in/david-boyne/
• GitHub: https://github.com/boyney123