What is Microservices?

Imagine a large, monolithic application as a giant ball of intertwined

spaghetti code. Microservices are the opposite - breaking down that
giant application into smaller, independent services, like individual
strands of spaghetti.
Microservices architecture is an architectural style that structures an
application as a collection of services that are:
 Loosely Coupled
 Fine-Grained
 Independently Deployable
 Communicate through APIs
There are a few microservivices patterns like:
 API Gateway/BFF which provides a single endpoint for client
applications to the microservices assigned to it.

 External Configuration Store which provides a common area

to store configuration information. This means there is one
source of the configuration values.
Lets compare microservices to monolith architecture
A monolith, in the simplest term, is a
single executable containing
business logic. This includes all the
supportive DLLs. When a monolith
deploys, functionality stops and is
replaced. Each service (or
component) in a monolith runs “in
process.” This means that each instance of the monolith has the
entire code base ready for instantiation.
With the microservice architecture, business logic is separated out
into out-of-process executables. This allows them to have many
instances of each running on different servers.

Benefits of microservices?
 Scalability: Individual services can be scaled up or down
independently based on their specific needs, allowing the
application to handle varying loads efficiently. The benefit of
 scalability
allows for the
number of
instances of
services to
differentiate between other services and a monolithic
application.
 Fault Isolation: When a monolith instance goes down, all
services in that instance also go down. There is no isolation of
services when failures occur. But with microservices
architecture, if one service fails, it typically won't bring down
the entire application. Other services can continue to function,
minimizing downtime and impact on users.

 Team Autonomy: By separating areas of concern in the

architecture, development teams can have autonomy from
other teams. This autonomy allows teams to develop and
deploy at a pace different than others. Monoliths typically
require the whole code base to be in the same language.
Because microservices are distinctly different applications,
they open the door to using different languages, allowing
flexibility in fitting the tool to the task at hand.
 Service Autonomy: Deployment of microservices is also a
benefit of service autonomy. Service autonomy is about
 separating concerns at the service layer. Services can be
implemented using different technologies and programming
languages, providing flexibility in technology choices.
 Data Autonomy: Each service can manage its own data
model, reducing complexity and potential data conflicts.
Having data isolated per microservice allows independent
changes to occur with minimal impact on others. With separate
databases, you also get the benefit of using differing data store
technologies.
Now, decentralizing data doesn’t store data in one central
location, but rather spread out across different databases or
storage systems. And because of this, decentralizing data aligns
well with microservices because each service can manage its own
data using the storage solution that best fits its needs.
Microservices and decentralized data are a powerful combination
that can improve the scalability, flexibility, and resilience of
software applications.

Building And Comunicating with microservices

 Interprocess Communication: Interprocess communication
(IPC) is crucial in a microservices architecture as it enables
microservices to communicate with each other efficiently.
Different IPC methods include REST and gRPC.
 API First Design: API first design involves designing the API
before the implementation. This approach ensures that the API
is well-defined and meets the requirements of all potential
consumers.
 Transport Mechanisms:
Two primary transport mechanisms for microservices
communication are REST and gRPC:
 o REST: REST (Representational State Transfer) uses
standard HTTP methods and is suitable for services
requiring web-scale, stateless interactions. It uses JSON
or XML for data exchange.
o gRPC: gRPC (Google Remote Procedure Call) is a high-
performance RPC framework that uses HTTP/2 for
transport, Protocol Buffers for message serialization, and
can generate client and server code in multiple languages.
It is ideal for low latency, high-throughput scenarios.
 Service Discovery: Service discovery is essential in a
microservices architecture to dynamically locate services. This
process ensures that microservices can find each other even as
they scale or move across different environments. Common
tools and protocols used for service discovery include:
o Consul: Provides service discovery, health checking, and
configuration management.
o Eureka: Developed by Netflix, it provides service
registry and discovery functionalities.
o Kubernetes: Includes built-in service discovery features
through DNS and environment variables

What is Decentralized Data?

In monolithic architectures, data management is centralized, which
can lead to bottlenecks and scalability issues. Decentralizing data is
crucial in microservices to ensure that each service operates
independently and efficiently. This shift involves moving from a
single, shared database to multiple, service-specific databases, each
tailored to the needs of the respective service.
There is a general “rule” stating that if a microservice persists data,
it should own that database. Each microservice exists to be the
processor of their business functionality, their domain. Each
microservice also needs to be the sole manager of the single source
of truth, the data. Without the ability to manage the single source of
truth, you risk losing data integrity.

Advantages of Decentralized Data

 Enhanced data ownership: Decentralization empowers
individuals or departments within an organization to own and
manage their data. This autonomy fosters innovation, as it
allows teams to tailor their data management practices to their
specific needs.
 Isolation: Data is isolated, reducing the risk of interference or
unintended consequences from other services.
 Improved scalability: Decentralized systems are inherently
scalable, as data can be distributed across multiple nodes. This
flexibility enables businesses to expand their operations
without facing the limitations of a centralized infrastructure.
 Resilience and fault tolerance: Decentralized data
architecture provides resilience against system failures. Even if
one node encounters issues, other nodes can continue to
function independently, ensuring business continuity and data
availability.
 Enhanced Security: A data breach in a centralized system
exposes everything at once. Decentralized storage makes
breaches less catastrophic, as attackers would only gain access
to a smaller portion of the data.
Overall, decentralizing data offers greater data ownership,
scalability, and security. However, it requires careful planning and
implementation to address the increased complexity.
Challenges of Decentralized Data
 Data consistency: Maintaining data consistency across
multiple decentralized nodes can be challenging.
Synchronization and version control mechanisms must be in
place to ensure that data remains accurate and up-to-date
across the network.
 Complex data integration: Integrating data from multiple
decentralized sources can be complex and time-consuming.
Data interoperability and compatibility become critical
considerations to ensure seamless data exchange between
different nodes.
 Increased security risks: With data dispersed across multiple
nodes, securing decentralized data becomes more intricate.
Each node must be adequately protected to prevent
unauthorized access or tampering. Robust encryption, access
controls, and authentication mechanisms are essential to
mitigate security risks effectively.

Examples of companies that embrace decentralized

data strategies:
Bitcoin (Blockchain), Ethereum (Blockchain), IPFS (InterPlanetary
File System), decentralizing data helps them approach enhances
security, immutability, distributed trust and eliminates the need for
intermediaries in various applications, promoting data availability
and fault tolerance.
Database Choices and Availability
Different types of databases can be used in a microservices
architecture, depending on the specific needs of each service:
 SQL Databases: Suitable for services requiring ACID
transactions and complex queries. Examples include MySQL,
PostgreSQL, and SQL Server.
 NoSQL Databases: Ideal for services needing high scalability
and flexible schema designs. Examples include MongoDB,
Cassandra, and Redis.
Ensuring high availability of data across distributed systems is
crucial, often achieved through replication and sharding strategies.
For instance, databases can be set up with primary and replica nodes
to ensure data redundancy and availability during failures.

Sharing data among microservices, particularly when

decentralizing data across different databases
One option is that we will have the shipping microservice call the
order microservice to retrieve the necessary data. As microservices
are calling each other to retrieve data, it is possible to have a rather
chatty network. As the network calls increase, so could the latency.
As the requests for data increase, so does the load on the related
databases.
The other option is to duplicate data. For instance, a parts
microservice manages its own database and serves as the single
source of truth for parts-related data. When an order microservice
requires part data, it only needs specific details relevant to its
function, not the entire dataset. Then you need to keep the
duplicated data synchronized across microservices, such as updating
the order microservice when a part’s description is corrected in the
parts microservice using messaging helps.

How to handle transactions

With data decentralized, we need to understand how to handle times
when the database is unavailable for the related microservice. This
introduces the CAP theorem: Consistency, Availability, and
Partition tolerance. The CAP theorem states that it is impossible to
provide more than two guarantees during a time of failure.
Network interruptions are easily possible so this leaves us with the
choice of Availability or Consistency. Either that it will continue
being available for callers when the microservice cannot reach its
database or the microservice must either return up-to- date
information or an error.

Managing transactions across microservices

In a microservices architecture, data is decentralized. This requires
analyzing business processes to understand their data needs and how
to keep the data in various microservices consistent:
 Some processes have activities that are sequential because of
their dependency on a flow of committed transactions.
  Other processes can be done in parallel, or a fan-out design,
as their activities are not dependent on other transactions.

The individual processes covered by the microservices still need to

complete before the business process as a whole is considered
complete and regardless of choice to use sequential or fan-out
design, the collection of transactions needs to be managed. This
introduces saga.

 Saga
A saga is a mechanism for managing potentially long-lived
transactions in a distributed system. Depends on the processing
needs of the transactions, there are 3 patterns:
 o Routing Slip:
This pattern is useful for sequential operations that require
preconditions to be met.
This involves passing information from one microservice to another.
Each microservice acts based on the information provided in the
routing slip. If a microservice encounters an error, it updates the
routing slip based on its information and sends it to the previous
microservice.
o Choreography:
Microservices communicate using events. Operations send events
upon completion, which other microservices receive and act upon
based on their subscriptions. In case of failures, microservices may
need compensating transactions to revert changes. But not all
processes require compensating transactions or data restoration to a
previous state.
o Orchestration:
Microservices are contacted using commands directly from the
central process manager. If a step fails, compensating transactions
are initiated.
This pattern is crucial for managing complex business processes
where a central controller is responsible for coordinating multiple
microservices to achieve a common goal, while also handling errors
and exceptions effectively.
  CQRS Pattern (Command Query Responsibility
Segregation)
CQRS is a good foundation for building decentralized systems. It is
a design pattern where the data models for writing (commands) and
reading (queries) are separated.
By splitting the microservice into two, each can be optimized for its
specific function, often using separate databases. The write database
is optimized for handling high volumes of incoming data, while the
read database is optimized with indexes for efficient data retrieval.
Event sourcing is great for processes handling financial transactions,
medical records, and even information for lawsuits. Also, it can
offset the role of the system.
 Event Soucing Pattern
This pattern is also a good fit for decentralized systems due to its
inherent properties.
This is a design pattern where changes to the state of an application
are stored as a sequence of events. Instead of only storing the
current state of data, it retains a history of all changes made to the
data. When an update occurs, the new value doesn’t overwrite the
previous one. Instead, an event representing this change is stored.
 Eventual Consistency
Eventual consistency is a common concept used in managing data
across microservices, which are essentially decentralized in nature.
This is a technique to ensure data eventually reaches a consistent
state across these decentralized stores.
Because microservices are independent applications communicates
using network, delays are expected. Therefore, designing
microservices with strong consistency can cause latency.
On the other hand, eventual consistency allows temporary
inconsistency after updates, but offers higher availability and
scalability. For most applications, eventual consistency's benefits
outweigh the drawbacks, especially for large-scale systems.
 Data Warehouse
A data warehouse is a large storage for various types of data from
multiple sources. This data is historical and can be used for
reporting and analysis. Data warehouses are different from OLTP
databases, which are used for transactions. Data warehouses use
OLAP databases, which are good for analytics. Data from OLTP
databases can be loaded into OLAP databases using ETL tools.
Microservices can also send data to data warehouses. Data
warehouses can be used to combine data from multiple sources into
a meterialized view that allow applications to retrieve predefined
views of data.
Transitioning from a monolithic system to
microservices architecture?
 Incremental Migration: This involves starting small by
identifying and extracting specific functionalities from the
monolith into independent microservices.
 Infrastructure Setup: Before diving into microservice
development, it's crucial to establish essential infrastructure
like CI/CD pipelines and a dedicated team to handle
development and management tasks.
 Code Separation and Refactoring: Analyze the monolith to
pinpoint functionalities suitable for microservices. Then,
refactor dependent code within the monolith to interact with
the new microservice through APIs.
 Database Management: Gradually migrate microservices to
their own independent databases to ensure data ownership and
integrity. Strategies like the Strangler Pattern and feature
flags can be used to manage this transition smoothly.
 Loose Coupling: Continuously strive to loosen dependencies
between the microservices and the monolith, aiming for
eventual microservice independence. This might involve
further splitting functionalities within the monolith if
necessary.
Conclusion
Decentralizing data is a critical aspect of microservices
architecture, enabling services to operate independently,
scale effectively, and maintain high availability and fault
tolerance. By adopting strategies like independent data
stores, service discovery, and eventual consistency,
organizations can overcome the limitations of monolithic
architectures and leverage the full potential of
microservices. As microservices continue to evolve,
emerging technologies and practices will further enhance
the capabilities and efficiency of decentralized data
management, paving the way for more resilient and
scalable systems.