SlideShare a Scribd company logo
Milano Apache Kafka Meetup
Milano Apache Kafka® Meetup / quantyca.it
www.quantyca.it
www.blindata.io
How to integrate legacy systems
within a modern polyglot and event
driven architecture using an
evolutionary strategy
Milano Apache Kafka® Meetup / quantyca.it
Milano, 4.07.2018
Agenda
Milano Apache Kafka® Meetup / quantyca.it
4
First steps in the
evolutionary journey
1
Why we need to evolve
3
The destination:
message driven
architecture
5
Legacy offloading
6
Attunity replicate
and the CDC maturity
model
2
The starting point:
spaghetti architecture
We really need to evolve?
Milano Apache Kafka® Meetup / quantyca.it
Martec’s Law
“The world is changing
very fast. Big will not beat
small anymore. It will be
the fast beating the slow.”
Rupert Murdoch
Evolve or die
Milano Apache Kafka® Meetup / quantyca.it
Agenda
Milano Apache Kafka® Meetup / quantyca.it
4
First steps in the
evolutionary journey
3
The destination:
message driven
architecture
5
Legacy offloading
6
Attunity replicate
and the CDC maturity
model
1
Why we need to evolve
2
The starting point:
spaghetti architecture
You are here: spaghetti architecture
Milano Apache Kafka® Meetup / quantyca.it
@see
Big Ball of Mud
Driving force: Polyglotism
9
Polyglot Programming
Applications should be written in a mix of
languages to take advantage of the fact that
different languages are suitable for tackling
different problems. Complex applications
combine different types of problems, so
picking the right language for the job may be
more productive than trying to fit all aspects
into a single language
2006
Polyglot Persistence
A complex enterprise application uses different
kinds of data, and already usually integrates
information from different sources. Increasingly
we'll see such applications manage their own
data using different technologies depending on
how the data is used. This trend will be
complementary to the trend of breaking up
application code into separate components
integrating through web services
2011
Milano Apache Kafka® Meetup / quantyca.it
Polyglotism without a clear IT strategy has increased the architectural technical debt
@see
Polyglot Persistence by Martin Fowler
Agenda
Milano Apache Kafka® Meetup / quantyca.it
4
First steps in the
evolutionary journey
3
The destination:
message driven
architecture
1
Why we need to evolve
5
Legacy offloading
6
Attunity replicate
and the CDC maturity
model
2
The starting point:
spaghetti architecture
Message driven systems to the rescue
Reactive Systems rely on asynchronous message-
passing to establish a boundary between components
that ensures loose coupling, isolation and location
transparency.
This boundary also provides the means to delegate
failures as messages. Employing explicit message-passing
enables load management, elasticity, and flow control by
shaping and monitoring the message queues in the
system and applying back-pressure when necessary.
Location transparent messaging as a means of
communication makes it possible for the management of
failure to work with the same constructs and semantics
across a cluster or within a single host. Non-blocking
communication allows recipients to only consume
resources while active, leading to less system overhead.
The Reactive Manifesto
Milano Apache Kafka® Meetup / quantyca.it
@see
https://www.reactivemanifesto.org/
Message driven architectures
Event-driven architectures (EDA) usually integrate several disparate systems together using message queues.
There are two common implementations of this type of architecture:
Milano Apache Kafka® Meetup / quantyca.it
broker pattern mediator pattern
@see
Variations in event-driven architecture: Find out if mediator or broker topology is right for you.
Traditional SOA (the evil) is an example of mediator pattern
In building communication structures between different processes, SOA approach put significant smarts into the
communication mechanism itself (smart pipe, dumb endpoints)
Milano Apache Kafka® Meetup / quantyca.it
=
Spaghetti architecture Spaghetti in a box architecture
@see
Does My Bus Look Big in This?
The real problem with traditional SOA
The problem with traditional SOA is not anyway the use of the mediator pattern but the degree to which developers
have deviated from bounded context. This is the key element that largely determines the quantum size and how much
damaging coupling appears in the overall architecture.
Enterprise services Share nothing domain services
>
coupling
continuum
Milano Apache Kafka® Meetup / quantyca.it
Traditional SOA Microservices
@see
Microservices as an Evolutionary Architecture
Software architectures aren’t created in a vacuum
If there are clear benefits, then why haven’t developers embraced this style before?
A decade ago, automatic provisioning of machines wasn’t possible. Operating systems were
commercial and licensed, with little support for automation. Real-world constraints like
budgets impact architectures, which is one of the reasons developers build more and more
elaborate shared resources architectures, segregated at the technical layers. If operations is
expensive and cumbersome, architects build around it, as they did in ESB-SOAs.
Milano Apache Kafka® Meetup / quantyca.it
Benefit and tradeoffs
Developers understand the benefits of everything and the
tradeoffs of nothing!
Rich Hickey, creator of Clojure
Milano Apache Kafka® Meetup / quantyca.it
Architects must understand both benefits and
tradeoffs of microservices and build engineering
practices accordingly.
Despite their popularity microservices are not the
right solution for any kind of problems
Alternative to microservices
Relaxing share nothing principle
A more commonly used architectural style for migration is a service-based architecture, which is similar to but could differ
from microservices in three important ways:
1. service granularity,
2. database scope,
3. integration middleware.
Service-based architectures are still domain-centric but address some challenges developers face when restructuring
existing applications toward more evolutionary architectures.
Microservices and service based architectures can cohesist. It’s a responsibility of the system architects to decide analyzing
the different use cases when to go with one solution and when to go with the other.
Milano Apache Kafka® Meetup / quantyca.it
@see
Service-Based Architecture as an Alternative to Microservice Architecture
Example of an hybrid service architecture
Milano Apache Kafka® Meetup / quantyca.it
Service based architecture and the service bus are used
when
1. control is more important than scalability
2. there are transaction that span bounded contexts
3. there are not enough skilled developers to implement
all with microservices
4. there is a monolith that requires an anti-corruption
layer
5. there is a legacy database that cannot be splitted
between microservices
Microservices architecture and the data bus are used in
all other cases
An API gateway hides to the callers if it are calling a service
through the service bus or the data bus
@see
Quantyca’s Reference Implementation
Agenda
Milano Apache Kafka® Meetup / quantyca.it
4
The starting point:
spaghetti architecture
3
The destination:
message driven
architecture
5
Legacy offloading
6
Attunity replicate
and the CDC maturity
model
1
Why we need to evolve
2
First steps in the
evolutionary journey
How to evolve from spaghetti architecture?
Milano Apache Kafka® Meetup / quantyca.it
Step 1: Working at the edge
Milano Apache Kafka® Meetup / quantyca.it
@see
Microservice Prerequisites
Step 2: Refactoring monoliths
Milano Apache Kafka® Meetup / quantyca.it
@see
How to break a Monolith into Microservices
Refactoring monoliths
Milano Apache Kafka® Meetup / quantyca.it
Agenda
Milano Apache Kafka® Meetup / quantyca.it
4
First steps in the
evolutionary journey
5
Legacy offloading
1
Why we need to evolve
3
The destination:
message driven
architecture
6
Attunity replicate
and the CDC maturity
model
2
The starting point:
spaghetti architecture
Vendor king antipattern
The viral lock-in
Many organizations become overambitious with some legacy software (i.e. ERP, Mainframe, ecc ..), leading to the vendor
king antipattern, an architecture built entirely around a vendor product that pathologically couples the
organization to a tool.
By placing an external tool or framework at the heart of the architecture, developers severely restrict their ability to
evolve in two key ways,
■ TECHNICAL PROCESS
From a business process standpoint the tool whole architecture depends by choices the vendor makes in
terms of persistence, supported infrastructure, and a host of other constraints
■ BUSINESS PROCESS
From a business process standpoint, the tool simply can’t support the optimal workflow; this is a side
effect of the Last 10% Trap. Most companies end up knuckling under the framework, modifying their
processes rather than trying to customize the tool. The more companies do that, the less differentiators
exist between companies, which is fine as long as that differentiation isn’t a competitive advantage.
Milano Apache Kafka® Meetup / quantyca.it
To big to remove
Milano Apache Kafka® Meetup / quantyca.it
Having developed and optimized Legacy King Systems for
decades, enterprises are looking for ways to capitalize on these
investments with the ability to reuse, repurpose, and integrate
them in support of new business and IT initiatives.
Legacy offloading
Milano Apache Kafka® Meetup / quantyca.it
DB2/MF
IMS
VSAM
SAP
Legacy King
Data
Warehouse
Straem
Analytics
Hybrid Cloud
Microservices
C-Level /
Menagement
Customer
LoB
Analyst
WHERE DATA NEED TO BE
KAFKA
OFFLOADING
Offloading via batch loading
● Queries based on timestamp orchestrated with custom ETL Jobs
● Consume significant processing power on the source system
● Usually executed during “batch windows” (not responsive)
● Unable to manage easily changes in the schema structure
● Custom error handling and failover management
Milano Apache Kafka® Meetup / quantyca.it
@see
Streaming change data capture
Offloading via CDC
Milano Apache Kafka® Meetup / quantyca.it
● Read incremental changes directly from the transaction log
● Minimize disruption to production workloads
● Fast updates (near real-time)
● Manage easily changes in the schema structure
● Mature market
@see
Streaming change data capture
Agenda
Milano Apache Kafka® Meetup / quantyca.it
4
First steps in the
evolutionary journey
3
The destination:
message driven
architecture
5
Legacy offloading
2
The starting point:
spaghetti architecture
1
Why we need to evolve
6
Attunity replicate
and the CDC maturity
model
Attunity Replicate
Milano Apache Kafka® Meetup / quantyca.it
@see
Attunity Replicate Home Page
Attunity: workflow
Milano Apache Kafka® Meetup / quantyca.it
STEP 1: Create connections STEP 2: Create task STEP 3: Define task
Attunity: workflow
Milano Apache Kafka® Meetup / quantyca.it
STEP 4: Run task
Attunity: Source and targets
Milano Apache Kafka® Meetup / quantyca.it
Attunity: Kafka target
Milano Apache Kafka® Meetup / quantyca.it
Attunity: business cases
Milano Apache Kafka® Meetup / quantyca.it
Offloading maturity model
Milano Apache Kafka® Meetup / quantyca.it
Data offloading Process offloading Business offloading
Thanks!
Questions?
Milano Apache Kafka® Meetup / quantyca.it
Milano, 4.07.2018
Corso Milano, 45 / 20900 Monza (MB)
T. +39 039 9000 210 / F. +39 039 9000 211 / @ info@quantyca.it
www.quantyca.it

More Related Content

What's hot (20)

Mulesoft Meetups - Salesforce & Mulesoft Integrations, Anypoint Security Poli...
Mulesoft Meetups - Salesforce & Mulesoft Integrations, Anypoint Security Poli...Mulesoft Meetups - Salesforce & Mulesoft Integrations, Anypoint Security Poli...
Mulesoft Meetups - Salesforce & Mulesoft Integrations, Anypoint Security Poli...
Ricardo Rodríguez
 
WSO2 API Manager 2.0 - Overview
WSO2 API Manager 2.0 - Overview WSO2 API Manager 2.0 - Overview
WSO2 API Manager 2.0 - Overview
Edgar Silva
 
Introduction to event based microservices
Introduction to event based microservicesIntroduction to event based microservices
Introduction to event based microservices
Grigoris Grigoriadis
 
API Security In Cloud Native Era
API Security In Cloud Native EraAPI Security In Cloud Native Era
API Security In Cloud Native Era
WSO2
 
[WSO2 API Manager Community Call] Expose Services as Managed APIs
[WSO2 API Manager Community Call] Expose Services as Managed APIs [WSO2 API Manager Community Call] Expose Services as Managed APIs
[WSO2 API Manager Community Call] Expose Services as Managed APIs
WSO2
 
apidays LIVE New York 2021 - Top 10 API security threats every API team shoul...
apidays LIVE New York 2021 - Top 10 API security threats every API team shoul...apidays LIVE New York 2021 - Top 10 API security threats every API team shoul...
apidays LIVE New York 2021 - Top 10 API security threats every API team shoul...
apidays
 
[APIdays Paris 2019] API Management in Service Mesh Using Istio and WSO2 API ...
[APIdays Paris 2019] API Management in Service Mesh Using Istio and WSO2 API ...[APIdays Paris 2019] API Management in Service Mesh Using Istio and WSO2 API ...
[APIdays Paris 2019] API Management in Service Mesh Using Istio and WSO2 API ...
WSO2
 
AlertSite Slideshow for the Booth at Web 2.0 Expo 2009
AlertSite Slideshow for the Booth at Web 2.0 Expo 2009AlertSite Slideshow for the Booth at Web 2.0 Expo 2009
AlertSite Slideshow for the Booth at Web 2.0 Expo 2009
AlertSite
 
API Management 101: The New API Experience with WSO2
API Management 101: The New API Experience with WSO2API Management 101: The New API Experience with WSO2
API Management 101: The New API Experience with WSO2
WSO2
 
Leveraging Async APIs to deliver cross domain agile collaboration
Leveraging Async APIs to deliver cross domain agile collaboration Leveraging Async APIs to deliver cross domain agile collaboration
Leveraging Async APIs to deliver cross domain agile collaboration
Nuwan Dias
 
Gravitee.io
Gravitee.ioGravitee.io
Gravitee.io
Knoldus Inc.
 
BI Studio profile
BI Studio profileBI Studio profile
BI Studio profile
Vassily Buzuyev
 
Stream Processing in Action
Stream Processing in ActionStream Processing in Action
Stream Processing in Action
WSO2
 
Explore the Latest on WSO2 Identity Server 5.11
Explore the Latest on WSO2 Identity Server 5.11Explore the Latest on WSO2 Identity Server 5.11
Explore the Latest on WSO2 Identity Server 5.11
WSO2
 
Meetup milano #3 all you need to know before creating your vpc
Meetup milano #3   all you need to know before creating your vpcMeetup milano #3   all you need to know before creating your vpc
Meetup milano #3 all you need to know before creating your vpc
Gonzalo Marcos Ansoain
 
Role of API Management in an API led Digital Economy
Role of API Management in an API led Digital EconomyRole of API Management in an API led Digital Economy
Role of API Management in an API led Digital Economy
WSO2
 
Rate Limiting GQLs Using Depth and Complexity Analysis
Rate Limiting GQLs Using Depth and Complexity AnalysisRate Limiting GQLs Using Depth and Complexity Analysis
Rate Limiting GQLs Using Depth and Complexity Analysis
WSO2
 
Building and Deploying Complex Streaming Data Processing Pipelines with WSO2 ...
Building and Deploying Complex Streaming Data Processing Pipelines with WSO2 ...Building and Deploying Complex Streaming Data Processing Pipelines with WSO2 ...
Building and Deploying Complex Streaming Data Processing Pipelines with WSO2 ...
WSO2
 
I Love APIs 2015: Advanced Security Extensions in Apigee Edge - JWT, JWE, JWS
I Love APIs 2015: Advanced Security Extensions in Apigee Edge - JWT, JWE, JWSI Love APIs 2015: Advanced Security Extensions in Apigee Edge - JWT, JWE, JWS
I Love APIs 2015: Advanced Security Extensions in Apigee Edge - JWT, JWE, JWS
Apigee | Google Cloud
 
[WSO2 API Day Toronto 2019] Cloud-native Integration for the Enterprise
[WSO2 API Day Toronto 2019] Cloud-native Integration for the Enterprise[WSO2 API Day Toronto 2019] Cloud-native Integration for the Enterprise
[WSO2 API Day Toronto 2019] Cloud-native Integration for the Enterprise
WSO2
 
Mulesoft Meetups - Salesforce & Mulesoft Integrations, Anypoint Security Poli...
Mulesoft Meetups - Salesforce & Mulesoft Integrations, Anypoint Security Poli...Mulesoft Meetups - Salesforce & Mulesoft Integrations, Anypoint Security Poli...
Mulesoft Meetups - Salesforce & Mulesoft Integrations, Anypoint Security Poli...
Ricardo Rodríguez
 
WSO2 API Manager 2.0 - Overview
WSO2 API Manager 2.0 - Overview WSO2 API Manager 2.0 - Overview
WSO2 API Manager 2.0 - Overview
Edgar Silva
 
Introduction to event based microservices
Introduction to event based microservicesIntroduction to event based microservices
Introduction to event based microservices
Grigoris Grigoriadis
 
API Security In Cloud Native Era
API Security In Cloud Native EraAPI Security In Cloud Native Era
API Security In Cloud Native Era
WSO2
 
[WSO2 API Manager Community Call] Expose Services as Managed APIs
[WSO2 API Manager Community Call] Expose Services as Managed APIs [WSO2 API Manager Community Call] Expose Services as Managed APIs
[WSO2 API Manager Community Call] Expose Services as Managed APIs
WSO2
 
apidays LIVE New York 2021 - Top 10 API security threats every API team shoul...
apidays LIVE New York 2021 - Top 10 API security threats every API team shoul...apidays LIVE New York 2021 - Top 10 API security threats every API team shoul...
apidays LIVE New York 2021 - Top 10 API security threats every API team shoul...
apidays
 
[APIdays Paris 2019] API Management in Service Mesh Using Istio and WSO2 API ...
[APIdays Paris 2019] API Management in Service Mesh Using Istio and WSO2 API ...[APIdays Paris 2019] API Management in Service Mesh Using Istio and WSO2 API ...
[APIdays Paris 2019] API Management in Service Mesh Using Istio and WSO2 API ...
WSO2
 
AlertSite Slideshow for the Booth at Web 2.0 Expo 2009
AlertSite Slideshow for the Booth at Web 2.0 Expo 2009AlertSite Slideshow for the Booth at Web 2.0 Expo 2009
AlertSite Slideshow for the Booth at Web 2.0 Expo 2009
AlertSite
 
API Management 101: The New API Experience with WSO2
API Management 101: The New API Experience with WSO2API Management 101: The New API Experience with WSO2
API Management 101: The New API Experience with WSO2
WSO2
 
Leveraging Async APIs to deliver cross domain agile collaboration
Leveraging Async APIs to deliver cross domain agile collaboration Leveraging Async APIs to deliver cross domain agile collaboration
Leveraging Async APIs to deliver cross domain agile collaboration
Nuwan Dias
 
Stream Processing in Action
Stream Processing in ActionStream Processing in Action
Stream Processing in Action
WSO2
 
Explore the Latest on WSO2 Identity Server 5.11
Explore the Latest on WSO2 Identity Server 5.11Explore the Latest on WSO2 Identity Server 5.11
Explore the Latest on WSO2 Identity Server 5.11
WSO2
 
Meetup milano #3 all you need to know before creating your vpc
Meetup milano #3   all you need to know before creating your vpcMeetup milano #3   all you need to know before creating your vpc
Meetup milano #3 all you need to know before creating your vpc
Gonzalo Marcos Ansoain
 
Role of API Management in an API led Digital Economy
Role of API Management in an API led Digital EconomyRole of API Management in an API led Digital Economy
Role of API Management in an API led Digital Economy
WSO2
 
Rate Limiting GQLs Using Depth and Complexity Analysis
Rate Limiting GQLs Using Depth and Complexity AnalysisRate Limiting GQLs Using Depth and Complexity Analysis
Rate Limiting GQLs Using Depth and Complexity Analysis
WSO2
 
Building and Deploying Complex Streaming Data Processing Pipelines with WSO2 ...
Building and Deploying Complex Streaming Data Processing Pipelines with WSO2 ...Building and Deploying Complex Streaming Data Processing Pipelines with WSO2 ...
Building and Deploying Complex Streaming Data Processing Pipelines with WSO2 ...
WSO2
 
I Love APIs 2015: Advanced Security Extensions in Apigee Edge - JWT, JWE, JWS
I Love APIs 2015: Advanced Security Extensions in Apigee Edge - JWT, JWE, JWSI Love APIs 2015: Advanced Security Extensions in Apigee Edge - JWT, JWE, JWS
I Love APIs 2015: Advanced Security Extensions in Apigee Edge - JWT, JWE, JWS
Apigee | Google Cloud
 
[WSO2 API Day Toronto 2019] Cloud-native Integration for the Enterprise
[WSO2 API Day Toronto 2019] Cloud-native Integration for the Enterprise[WSO2 API Day Toronto 2019] Cloud-native Integration for the Enterprise
[WSO2 API Day Toronto 2019] Cloud-native Integration for the Enterprise
WSO2
 

Similar to How to integrate legacy systems within a modern polyglot and event driven architecture using an evolutionary strategy (20)

Message Driven and Event Sourcing
Message Driven and Event SourcingMessage Driven and Event Sourcing
Message Driven and Event Sourcing
Paolo Castagna
 
Microservices Architecture, Monolith Migration Patterns
Microservices Architecture, Monolith Migration PatternsMicroservices Architecture, Monolith Migration Patterns
Microservices Architecture, Monolith Migration Patterns
Araf Karsh Hamid
 
Model-Driven Development and Operation of Multi-Cloud Applications: The MODAC...
Model-Driven Development and Operation of Multi-Cloud Applications: The MODAC...Model-Driven Development and Operation of Multi-Cloud Applications: The MODAC...
Model-Driven Development and Operation of Multi-Cloud Applications: The MODAC...
korthmerlesb
 
Do you need microservices architecture?
Do you need microservices architecture?Do you need microservices architecture?
Do you need microservices architecture?
Manu Pk
 
A Guide on What Are Microservices: Pros, Cons, Use Cases, and More
A Guide on What Are Microservices: Pros, Cons, Use Cases, and MoreA Guide on What Are Microservices: Pros, Cons, Use Cases, and More
A Guide on What Are Microservices: Pros, Cons, Use Cases, and More
Simform
 
Microservices Architecture for e-Commerce
Microservices Architecture for e-CommerceMicroservices Architecture for e-Commerce
Microservices Architecture for e-Commerce
Divante
 
Microservices.pdf
Microservices.pdfMicroservices.pdf
Microservices.pdf
SelmaJelovac1
 
Microservices Design Principles.pdf
Microservices Design Principles.pdfMicroservices Design Principles.pdf
Microservices Design Principles.pdf
Simform
 
Microservices: Aren't Microservices Just SOA?
Microservices: Aren't Microservices Just SOA?Microservices: Aren't Microservices Just SOA?
Microservices: Aren't Microservices Just SOA?
Chris Sterling
 
Microservices: A Step Towards Modernizing Healthcare Applications
Microservices: A Step Towards Modernizing Healthcare ApplicationsMicroservices: A Step Towards Modernizing Healthcare Applications
Microservices: A Step Towards Modernizing Healthcare Applications
CitiusTech
 
FEVR - Micro Frontend
FEVR - Micro FrontendFEVR - Micro Frontend
FEVR - Micro Frontend
Miki Lombardi
 
Over view of software artitecture
Over view of software artitectureOver view of software artitecture
Over view of software artitecture
ABDEL RAHMAN KARIM
 
Distributed Systems in Data Engineering
Distributed Systems in Data EngineeringDistributed Systems in Data Engineering
Distributed Systems in Data Engineering
Oluwasegun Matthew
 
Microservices: Why and When? - Alon Fliess, CodeValue - Cloud Native Day Tel ...
Microservices: Why and When? - Alon Fliess, CodeValue - Cloud Native Day Tel ...Microservices: Why and When? - Alon Fliess, CodeValue - Cloud Native Day Tel ...
Microservices: Why and When? - Alon Fliess, CodeValue - Cloud Native Day Tel ...
Cloud Native Day Tel Aviv
 
Micro Front-End & Microservices - Plansoft
Micro Front-End & Microservices - PlansoftMicro Front-End & Microservices - Plansoft
Micro Front-End & Microservices - Plansoft
Miki Lombardi
 
Introducción a Microservicios, SUSE CaaS Platform y Kubernetes
Introducción a Microservicios, SUSE CaaS Platform y KubernetesIntroducción a Microservicios, SUSE CaaS Platform y Kubernetes
Introducción a Microservicios, SUSE CaaS Platform y Kubernetes
SUSE España
 
Microservices architecture
Microservices architectureMicroservices architecture
Microservices architecture
Faren faren
 
Embracing Containers and Microservices for Future Proof Application Moderniza...
Embracing Containers and Microservices for Future Proof Application Moderniza...Embracing Containers and Microservices for Future Proof Application Moderniza...
Embracing Containers and Microservices for Future Proof Application Moderniza...
Marlabs
 
20161220 - microservice
20161220 - microservice20161220 - microservice
20161220 - microservice
Jamie (Taka) Wang
 
Beyond 12 Factor - Developing Cloud Native Applications
Beyond 12 Factor - Developing Cloud Native ApplicationsBeyond 12 Factor - Developing Cloud Native Applications
Beyond 12 Factor - Developing Cloud Native Applications
Container Solutions
 
Message Driven and Event Sourcing
Message Driven and Event SourcingMessage Driven and Event Sourcing
Message Driven and Event Sourcing
Paolo Castagna
 
Microservices Architecture, Monolith Migration Patterns
Microservices Architecture, Monolith Migration PatternsMicroservices Architecture, Monolith Migration Patterns
Microservices Architecture, Monolith Migration Patterns
Araf Karsh Hamid
 
Model-Driven Development and Operation of Multi-Cloud Applications: The MODAC...
Model-Driven Development and Operation of Multi-Cloud Applications: The MODAC...Model-Driven Development and Operation of Multi-Cloud Applications: The MODAC...
Model-Driven Development and Operation of Multi-Cloud Applications: The MODAC...
korthmerlesb
 
Do you need microservices architecture?
Do you need microservices architecture?Do you need microservices architecture?
Do you need microservices architecture?
Manu Pk
 
A Guide on What Are Microservices: Pros, Cons, Use Cases, and More
A Guide on What Are Microservices: Pros, Cons, Use Cases, and MoreA Guide on What Are Microservices: Pros, Cons, Use Cases, and More
A Guide on What Are Microservices: Pros, Cons, Use Cases, and More
Simform
 
Microservices Architecture for e-Commerce
Microservices Architecture for e-CommerceMicroservices Architecture for e-Commerce
Microservices Architecture for e-Commerce
Divante
 
Microservices Design Principles.pdf
Microservices Design Principles.pdfMicroservices Design Principles.pdf
Microservices Design Principles.pdf
Simform
 
Microservices: Aren't Microservices Just SOA?
Microservices: Aren't Microservices Just SOA?Microservices: Aren't Microservices Just SOA?
Microservices: Aren't Microservices Just SOA?
Chris Sterling
 
Microservices: A Step Towards Modernizing Healthcare Applications
Microservices: A Step Towards Modernizing Healthcare ApplicationsMicroservices: A Step Towards Modernizing Healthcare Applications
Microservices: A Step Towards Modernizing Healthcare Applications
CitiusTech
 
FEVR - Micro Frontend
FEVR - Micro FrontendFEVR - Micro Frontend
FEVR - Micro Frontend
Miki Lombardi
 
Over view of software artitecture
Over view of software artitectureOver view of software artitecture
Over view of software artitecture
ABDEL RAHMAN KARIM
 
Distributed Systems in Data Engineering
Distributed Systems in Data EngineeringDistributed Systems in Data Engineering
Distributed Systems in Data Engineering
Oluwasegun Matthew
 
Microservices: Why and When? - Alon Fliess, CodeValue - Cloud Native Day Tel ...
Microservices: Why and When? - Alon Fliess, CodeValue - Cloud Native Day Tel ...Microservices: Why and When? - Alon Fliess, CodeValue - Cloud Native Day Tel ...
Microservices: Why and When? - Alon Fliess, CodeValue - Cloud Native Day Tel ...
Cloud Native Day Tel Aviv
 
Micro Front-End & Microservices - Plansoft
Micro Front-End & Microservices - PlansoftMicro Front-End & Microservices - Plansoft
Micro Front-End & Microservices - Plansoft
Miki Lombardi
 
Introducción a Microservicios, SUSE CaaS Platform y Kubernetes
Introducción a Microservicios, SUSE CaaS Platform y KubernetesIntroducción a Microservicios, SUSE CaaS Platform y Kubernetes
Introducción a Microservicios, SUSE CaaS Platform y Kubernetes
SUSE España
 
Microservices architecture
Microservices architectureMicroservices architecture
Microservices architecture
Faren faren
 
Embracing Containers and Microservices for Future Proof Application Moderniza...
Embracing Containers and Microservices for Future Proof Application Moderniza...Embracing Containers and Microservices for Future Proof Application Moderniza...
Embracing Containers and Microservices for Future Proof Application Moderniza...
Marlabs
 
Beyond 12 Factor - Developing Cloud Native Applications
Beyond 12 Factor - Developing Cloud Native ApplicationsBeyond 12 Factor - Developing Cloud Native Applications
Beyond 12 Factor - Developing Cloud Native Applications
Container Solutions
 
Ad

More from Andrea Gioia (11)

DATA & DRINKS: Data Management Trends.pdf
DATA & DRINKS: Data Management Trends.pdfDATA & DRINKS: Data Management Trends.pdf
DATA & DRINKS: Data Management Trends.pdf
Andrea Gioia
 
Kafka Summit 2022: Handling Eventual Consistency in a Transactional World.pdf
Kafka Summit 2022: Handling Eventual Consistency in a Transactional World.pdfKafka Summit 2022: Handling Eventual Consistency in a Transactional World.pdf
Kafka Summit 2022: Handling Eventual Consistency in a Transactional World.pdf
Andrea Gioia
 
KAFKA Summit 2021: From legacy systems to microservices and back.pdf
KAFKA Summit 2021: From legacy systems to microservices and back.pdfKAFKA Summit 2021: From legacy systems to microservices and back.pdf
KAFKA Summit 2021: From legacy systems to microservices and back.pdf
Andrea Gioia
 
Ripartire dai dati ponendo le integrazioni al centro della propria strategia.pdf
Ripartire dai dati ponendo le integrazioni al centro della propria strategia.pdfRipartire dai dati ponendo le integrazioni al centro della propria strategia.pdf
Ripartire dai dati ponendo le integrazioni al centro della propria strategia.pdf
Andrea Gioia
 
I Software passano, i Dati restano.pdf
I Software passano, i Dati restano.pdfI Software passano, i Dati restano.pdf
I Software passano, i Dati restano.pdf
Andrea Gioia
 
THE DATA JOURNEY: FROM CHAOS TO STRATEGY .pdf
THE DATA JOURNEY: FROM CHAOS TO STRATEGY .pdfTHE DATA JOURNEY: FROM CHAOS TO STRATEGY .pdf
THE DATA JOURNEY: FROM CHAOS TO STRATEGY .pdf
Andrea Gioia
 
IT matters once again
IT matters once againIT matters once again
IT matters once again
Andrea Gioia
 
Framework tecnologici per i Big Data: Data Lake & Data River
Framework tecnologici per i Big Data: Data Lake & Data RiverFramework tecnologici per i Big Data: Data Lake & Data River
Framework tecnologici per i Big Data: Data Lake & Data River
Andrea Gioia
 
Fast data platforms - Hadoop User Group (Italy)
Fast data platforms  - Hadoop User Group (Italy)Fast data platforms  - Hadoop User Group (Italy)
Fast data platforms - Hadoop User Group (Italy)
Andrea Gioia
 
Open Source Location Intelligence with SpagoBI
Open Source Location Intelligence with SpagoBIOpen Source Location Intelligence with SpagoBI
Open Source Location Intelligence with SpagoBI
Andrea Gioia
 
Corso sistemi aperti - Laboratorio - Case Study (SpagoBI)
Corso sistemi aperti - Laboratorio - Case Study (SpagoBI)Corso sistemi aperti - Laboratorio - Case Study (SpagoBI)
Corso sistemi aperti - Laboratorio - Case Study (SpagoBI)
Andrea Gioia
 
DATA & DRINKS: Data Management Trends.pdf
DATA & DRINKS: Data Management Trends.pdfDATA & DRINKS: Data Management Trends.pdf
DATA & DRINKS: Data Management Trends.pdf
Andrea Gioia
 
Kafka Summit 2022: Handling Eventual Consistency in a Transactional World.pdf
Kafka Summit 2022: Handling Eventual Consistency in a Transactional World.pdfKafka Summit 2022: Handling Eventual Consistency in a Transactional World.pdf
Kafka Summit 2022: Handling Eventual Consistency in a Transactional World.pdf
Andrea Gioia
 
KAFKA Summit 2021: From legacy systems to microservices and back.pdf
KAFKA Summit 2021: From legacy systems to microservices and back.pdfKAFKA Summit 2021: From legacy systems to microservices and back.pdf
KAFKA Summit 2021: From legacy systems to microservices and back.pdf
Andrea Gioia
 
Ripartire dai dati ponendo le integrazioni al centro della propria strategia.pdf
Ripartire dai dati ponendo le integrazioni al centro della propria strategia.pdfRipartire dai dati ponendo le integrazioni al centro della propria strategia.pdf
Ripartire dai dati ponendo le integrazioni al centro della propria strategia.pdf
Andrea Gioia
 
I Software passano, i Dati restano.pdf
I Software passano, i Dati restano.pdfI Software passano, i Dati restano.pdf
I Software passano, i Dati restano.pdf
Andrea Gioia
 
THE DATA JOURNEY: FROM CHAOS TO STRATEGY .pdf
THE DATA JOURNEY: FROM CHAOS TO STRATEGY .pdfTHE DATA JOURNEY: FROM CHAOS TO STRATEGY .pdf
THE DATA JOURNEY: FROM CHAOS TO STRATEGY .pdf
Andrea Gioia
 
IT matters once again
IT matters once againIT matters once again
IT matters once again
Andrea Gioia
 
Framework tecnologici per i Big Data: Data Lake & Data River
Framework tecnologici per i Big Data: Data Lake & Data RiverFramework tecnologici per i Big Data: Data Lake & Data River
Framework tecnologici per i Big Data: Data Lake & Data River
Andrea Gioia
 
Fast data platforms - Hadoop User Group (Italy)
Fast data platforms  - Hadoop User Group (Italy)Fast data platforms  - Hadoop User Group (Italy)
Fast data platforms - Hadoop User Group (Italy)
Andrea Gioia
 
Open Source Location Intelligence with SpagoBI
Open Source Location Intelligence with SpagoBIOpen Source Location Intelligence with SpagoBI
Open Source Location Intelligence with SpagoBI
Andrea Gioia
 
Corso sistemi aperti - Laboratorio - Case Study (SpagoBI)
Corso sistemi aperti - Laboratorio - Case Study (SpagoBI)Corso sistemi aperti - Laboratorio - Case Study (SpagoBI)
Corso sistemi aperti - Laboratorio - Case Study (SpagoBI)
Andrea Gioia
 
Ad

Recently uploaded (20)

Software Engineering Project Presentation Tanisha Tasnuva
Software Engineering Project Presentation Tanisha TasnuvaSoftware Engineering Project Presentation Tanisha Tasnuva
Software Engineering Project Presentation Tanisha Tasnuva
tanishatasnuva76
 
Video Games and Artificial-Realities.pptx
Video Games and Artificial-Realities.pptxVideo Games and Artificial-Realities.pptx
Video Games and Artificial-Realities.pptx
HadiBadri1
 
Axial Capacity Estimation of FRP-strengthened Corroded Concrete Columns
Axial Capacity Estimation of FRP-strengthened Corroded Concrete ColumnsAxial Capacity Estimation of FRP-strengthened Corroded Concrete Columns
Axial Capacity Estimation of FRP-strengthened Corroded Concrete Columns
Journal of Soft Computing in Civil Engineering
 
Highway Engineering - Pavement materials
Highway Engineering - Pavement materialsHighway Engineering - Pavement materials
Highway Engineering - Pavement materials
AmrutaBhosale9
 
Fresh concrete Workability Measurement
Fresh concrete  Workability  MeasurementFresh concrete  Workability  Measurement
Fresh concrete Workability Measurement
SasiVarman5
 
UNIT-1-PPT-Introduction about Power System Operation and Control
UNIT-1-PPT-Introduction about Power System Operation and ControlUNIT-1-PPT-Introduction about Power System Operation and Control
UNIT-1-PPT-Introduction about Power System Operation and Control
Sridhar191373
 
ISO 5011 Air Filter Catalogues .pdf
ISO 5011 Air Filter Catalogues      .pdfISO 5011 Air Filter Catalogues      .pdf
ISO 5011 Air Filter Catalogues .pdf
FILTRATION ENGINEERING & CUNSULTANT
 
Numerical Investigation of the Aerodynamic Characteristics for a Darrieus H-t...
Numerical Investigation of the Aerodynamic Characteristics for a Darrieus H-t...Numerical Investigation of the Aerodynamic Characteristics for a Darrieus H-t...
Numerical Investigation of the Aerodynamic Characteristics for a Darrieus H-t...
Mohamed905031
 
Utilizing Biomedical Waste for Sustainable Brick Manufacturing: A Novel Appro...
Utilizing Biomedical Waste for Sustainable Brick Manufacturing: A Novel Appro...Utilizing Biomedical Waste for Sustainable Brick Manufacturing: A Novel Appro...
Utilizing Biomedical Waste for Sustainable Brick Manufacturing: A Novel Appro...
IRJET Journal
 
ISO 4548-7 Filter Vibration Fatigue Test Rig Catalogue.pdf
ISO 4548-7 Filter Vibration Fatigue Test Rig Catalogue.pdfISO 4548-7 Filter Vibration Fatigue Test Rig Catalogue.pdf
ISO 4548-7 Filter Vibration Fatigue Test Rig Catalogue.pdf
FILTRATION ENGINEERING & CUNSULTANT
 
Influence line diagram in a robust model
Influence line diagram in a robust modelInfluence line diagram in a robust model
Influence line diagram in a robust model
ParthaSengupta26
 
May 2025: Top 10 Cited Articles in Software Engineering & Applications Intern...
May 2025: Top 10 Cited Articles in Software Engineering & Applications Intern...May 2025: Top 10 Cited Articles in Software Engineering & Applications Intern...
May 2025: Top 10 Cited Articles in Software Engineering & Applications Intern...
sebastianku31
 
Air Filter Flat Sheet Media-Catalouge-Final.pdf
Air Filter Flat Sheet Media-Catalouge-Final.pdfAir Filter Flat Sheet Media-Catalouge-Final.pdf
Air Filter Flat Sheet Media-Catalouge-Final.pdf
FILTRATION ENGINEERING & CUNSULTANT
 
Software_Engineering_in_6_Hours_lyst1728638742594.pdf
Software_Engineering_in_6_Hours_lyst1728638742594.pdfSoftware_Engineering_in_6_Hours_lyst1728638742594.pdf
Software_Engineering_in_6_Hours_lyst1728638742594.pdf
VanshMunjal7
 
Application Security and Secure Software Development Lifecycle
Application  Security and Secure Software Development LifecycleApplication  Security and Secure Software Development Lifecycle
Application Security and Secure Software Development Lifecycle
DrKavithaP1
 
Proposed EPA Municipal Waste Combustor Rule
Proposed EPA Municipal Waste Combustor RuleProposed EPA Municipal Waste Combustor Rule
Proposed EPA Municipal Waste Combustor Rule
AlvaroLinero2
 
Android basics – Key Codes – ADB – Rooting Android – Boot Process – File Syst...
Android basics – Key Codes – ADB – Rooting Android – Boot Process – File Syst...Android basics – Key Codes – ADB – Rooting Android – Boot Process – File Syst...
Android basics – Key Codes – ADB – Rooting Android – Boot Process – File Syst...
ManiMaran230751
 
world subdivision.pdf...................
world subdivision.pdf...................world subdivision.pdf...................
world subdivision.pdf...................
bmmederos10
 
Software Developer Portfolio: Backend Architecture & Performance Optimization
Software Developer Portfolio: Backend Architecture & Performance OptimizationSoftware Developer Portfolio: Backend Architecture & Performance Optimization
Software Developer Portfolio: Backend Architecture & Performance Optimization
kiwoong (daniel) kim
 
ENERGY STORING DEVICES-Primary Battery.pdf
ENERGY STORING DEVICES-Primary Battery.pdfENERGY STORING DEVICES-Primary Battery.pdf
ENERGY STORING DEVICES-Primary Battery.pdf
TAMILISAI R
 
Software Engineering Project Presentation Tanisha Tasnuva
Software Engineering Project Presentation Tanisha TasnuvaSoftware Engineering Project Presentation Tanisha Tasnuva
Software Engineering Project Presentation Tanisha Tasnuva
tanishatasnuva76
 
Video Games and Artificial-Realities.pptx
Video Games and Artificial-Realities.pptxVideo Games and Artificial-Realities.pptx
Video Games and Artificial-Realities.pptx
HadiBadri1
 
Highway Engineering - Pavement materials
Highway Engineering - Pavement materialsHighway Engineering - Pavement materials
Highway Engineering - Pavement materials
AmrutaBhosale9
 
Fresh concrete Workability Measurement
Fresh concrete  Workability  MeasurementFresh concrete  Workability  Measurement
Fresh concrete Workability Measurement
SasiVarman5
 
UNIT-1-PPT-Introduction about Power System Operation and Control
UNIT-1-PPT-Introduction about Power System Operation and ControlUNIT-1-PPT-Introduction about Power System Operation and Control
UNIT-1-PPT-Introduction about Power System Operation and Control
Sridhar191373
 
Numerical Investigation of the Aerodynamic Characteristics for a Darrieus H-t...
Numerical Investigation of the Aerodynamic Characteristics for a Darrieus H-t...Numerical Investigation of the Aerodynamic Characteristics for a Darrieus H-t...
Numerical Investigation of the Aerodynamic Characteristics for a Darrieus H-t...
Mohamed905031
 
Utilizing Biomedical Waste for Sustainable Brick Manufacturing: A Novel Appro...
Utilizing Biomedical Waste for Sustainable Brick Manufacturing: A Novel Appro...Utilizing Biomedical Waste for Sustainable Brick Manufacturing: A Novel Appro...
Utilizing Biomedical Waste for Sustainable Brick Manufacturing: A Novel Appro...
IRJET Journal
 
Influence line diagram in a robust model
Influence line diagram in a robust modelInfluence line diagram in a robust model
Influence line diagram in a robust model
ParthaSengupta26
 
May 2025: Top 10 Cited Articles in Software Engineering & Applications Intern...
May 2025: Top 10 Cited Articles in Software Engineering & Applications Intern...May 2025: Top 10 Cited Articles in Software Engineering & Applications Intern...
May 2025: Top 10 Cited Articles in Software Engineering & Applications Intern...
sebastianku31
 
Software_Engineering_in_6_Hours_lyst1728638742594.pdf
Software_Engineering_in_6_Hours_lyst1728638742594.pdfSoftware_Engineering_in_6_Hours_lyst1728638742594.pdf
Software_Engineering_in_6_Hours_lyst1728638742594.pdf
VanshMunjal7
 
Application Security and Secure Software Development Lifecycle
Application  Security and Secure Software Development LifecycleApplication  Security and Secure Software Development Lifecycle
Application Security and Secure Software Development Lifecycle
DrKavithaP1
 
Proposed EPA Municipal Waste Combustor Rule
Proposed EPA Municipal Waste Combustor RuleProposed EPA Municipal Waste Combustor Rule
Proposed EPA Municipal Waste Combustor Rule
AlvaroLinero2
 
Android basics – Key Codes – ADB – Rooting Android – Boot Process – File Syst...
Android basics – Key Codes – ADB – Rooting Android – Boot Process – File Syst...Android basics – Key Codes – ADB – Rooting Android – Boot Process – File Syst...
Android basics – Key Codes – ADB – Rooting Android – Boot Process – File Syst...
ManiMaran230751
 
world subdivision.pdf...................
world subdivision.pdf...................world subdivision.pdf...................
world subdivision.pdf...................
bmmederos10
 
Software Developer Portfolio: Backend Architecture & Performance Optimization
Software Developer Portfolio: Backend Architecture & Performance OptimizationSoftware Developer Portfolio: Backend Architecture & Performance Optimization
Software Developer Portfolio: Backend Architecture & Performance Optimization
kiwoong (daniel) kim
 
ENERGY STORING DEVICES-Primary Battery.pdf
ENERGY STORING DEVICES-Primary Battery.pdfENERGY STORING DEVICES-Primary Battery.pdf
ENERGY STORING DEVICES-Primary Battery.pdf
TAMILISAI R
 

How to integrate legacy systems within a modern polyglot and event driven architecture using an evolutionary strategy

  • 2. Milano Apache Kafka® Meetup / quantyca.it www.quantyca.it www.blindata.io
  • 3. How to integrate legacy systems within a modern polyglot and event driven architecture using an evolutionary strategy Milano Apache Kafka® Meetup / quantyca.it Milano, 4.07.2018
  • 4. Agenda Milano Apache Kafka® Meetup / quantyca.it 4 First steps in the evolutionary journey 1 Why we need to evolve 3 The destination: message driven architecture 5 Legacy offloading 6 Attunity replicate and the CDC maturity model 2 The starting point: spaghetti architecture
  • 5. We really need to evolve? Milano Apache Kafka® Meetup / quantyca.it Martec’s Law “The world is changing very fast. Big will not beat small anymore. It will be the fast beating the slow.” Rupert Murdoch
  • 6. Evolve or die Milano Apache Kafka® Meetup / quantyca.it
  • 7. Agenda Milano Apache Kafka® Meetup / quantyca.it 4 First steps in the evolutionary journey 3 The destination: message driven architecture 5 Legacy offloading 6 Attunity replicate and the CDC maturity model 1 Why we need to evolve 2 The starting point: spaghetti architecture
  • 8. You are here: spaghetti architecture Milano Apache Kafka® Meetup / quantyca.it @see Big Ball of Mud
  • 9. Driving force: Polyglotism 9 Polyglot Programming Applications should be written in a mix of languages to take advantage of the fact that different languages are suitable for tackling different problems. Complex applications combine different types of problems, so picking the right language for the job may be more productive than trying to fit all aspects into a single language 2006 Polyglot Persistence A complex enterprise application uses different kinds of data, and already usually integrates information from different sources. Increasingly we'll see such applications manage their own data using different technologies depending on how the data is used. This trend will be complementary to the trend of breaking up application code into separate components integrating through web services 2011 Milano Apache Kafka® Meetup / quantyca.it Polyglotism without a clear IT strategy has increased the architectural technical debt @see Polyglot Persistence by Martin Fowler
  • 10. Agenda Milano Apache Kafka® Meetup / quantyca.it 4 First steps in the evolutionary journey 3 The destination: message driven architecture 1 Why we need to evolve 5 Legacy offloading 6 Attunity replicate and the CDC maturity model 2 The starting point: spaghetti architecture
  • 11. Message driven systems to the rescue Reactive Systems rely on asynchronous message- passing to establish a boundary between components that ensures loose coupling, isolation and location transparency. This boundary also provides the means to delegate failures as messages. Employing explicit message-passing enables load management, elasticity, and flow control by shaping and monitoring the message queues in the system and applying back-pressure when necessary. Location transparent messaging as a means of communication makes it possible for the management of failure to work with the same constructs and semantics across a cluster or within a single host. Non-blocking communication allows recipients to only consume resources while active, leading to less system overhead. The Reactive Manifesto Milano Apache Kafka® Meetup / quantyca.it @see https://www.reactivemanifesto.org/
  • 12. Message driven architectures Event-driven architectures (EDA) usually integrate several disparate systems together using message queues. There are two common implementations of this type of architecture: Milano Apache Kafka® Meetup / quantyca.it broker pattern mediator pattern @see Variations in event-driven architecture: Find out if mediator or broker topology is right for you.
  • 13. Traditional SOA (the evil) is an example of mediator pattern In building communication structures between different processes, SOA approach put significant smarts into the communication mechanism itself (smart pipe, dumb endpoints) Milano Apache Kafka® Meetup / quantyca.it = Spaghetti architecture Spaghetti in a box architecture @see Does My Bus Look Big in This?
  • 14. The real problem with traditional SOA The problem with traditional SOA is not anyway the use of the mediator pattern but the degree to which developers have deviated from bounded context. This is the key element that largely determines the quantum size and how much damaging coupling appears in the overall architecture. Enterprise services Share nothing domain services > coupling continuum Milano Apache Kafka® Meetup / quantyca.it Traditional SOA Microservices @see Microservices as an Evolutionary Architecture
  • 15. Software architectures aren’t created in a vacuum If there are clear benefits, then why haven’t developers embraced this style before? A decade ago, automatic provisioning of machines wasn’t possible. Operating systems were commercial and licensed, with little support for automation. Real-world constraints like budgets impact architectures, which is one of the reasons developers build more and more elaborate shared resources architectures, segregated at the technical layers. If operations is expensive and cumbersome, architects build around it, as they did in ESB-SOAs. Milano Apache Kafka® Meetup / quantyca.it
  • 16. Benefit and tradeoffs Developers understand the benefits of everything and the tradeoffs of nothing! Rich Hickey, creator of Clojure Milano Apache Kafka® Meetup / quantyca.it Architects must understand both benefits and tradeoffs of microservices and build engineering practices accordingly. Despite their popularity microservices are not the right solution for any kind of problems
  • 17. Alternative to microservices Relaxing share nothing principle A more commonly used architectural style for migration is a service-based architecture, which is similar to but could differ from microservices in three important ways: 1. service granularity, 2. database scope, 3. integration middleware. Service-based architectures are still domain-centric but address some challenges developers face when restructuring existing applications toward more evolutionary architectures. Microservices and service based architectures can cohesist. It’s a responsibility of the system architects to decide analyzing the different use cases when to go with one solution and when to go with the other. Milano Apache Kafka® Meetup / quantyca.it @see Service-Based Architecture as an Alternative to Microservice Architecture
  • 18. Example of an hybrid service architecture Milano Apache Kafka® Meetup / quantyca.it Service based architecture and the service bus are used when 1. control is more important than scalability 2. there are transaction that span bounded contexts 3. there are not enough skilled developers to implement all with microservices 4. there is a monolith that requires an anti-corruption layer 5. there is a legacy database that cannot be splitted between microservices Microservices architecture and the data bus are used in all other cases An API gateway hides to the callers if it are calling a service through the service bus or the data bus @see Quantyca’s Reference Implementation
  • 19. Agenda Milano Apache Kafka® Meetup / quantyca.it 4 The starting point: spaghetti architecture 3 The destination: message driven architecture 5 Legacy offloading 6 Attunity replicate and the CDC maturity model 1 Why we need to evolve 2 First steps in the evolutionary journey
  • 20. How to evolve from spaghetti architecture? Milano Apache Kafka® Meetup / quantyca.it
  • 21. Step 1: Working at the edge Milano Apache Kafka® Meetup / quantyca.it @see Microservice Prerequisites
  • 22. Step 2: Refactoring monoliths Milano Apache Kafka® Meetup / quantyca.it @see How to break a Monolith into Microservices
  • 23. Refactoring monoliths Milano Apache Kafka® Meetup / quantyca.it
  • 24. Agenda Milano Apache Kafka® Meetup / quantyca.it 4 First steps in the evolutionary journey 5 Legacy offloading 1 Why we need to evolve 3 The destination: message driven architecture 6 Attunity replicate and the CDC maturity model 2 The starting point: spaghetti architecture
  • 25. Vendor king antipattern The viral lock-in Many organizations become overambitious with some legacy software (i.e. ERP, Mainframe, ecc ..), leading to the vendor king antipattern, an architecture built entirely around a vendor product that pathologically couples the organization to a tool. By placing an external tool or framework at the heart of the architecture, developers severely restrict their ability to evolve in two key ways, ■ TECHNICAL PROCESS From a business process standpoint the tool whole architecture depends by choices the vendor makes in terms of persistence, supported infrastructure, and a host of other constraints ■ BUSINESS PROCESS From a business process standpoint, the tool simply can’t support the optimal workflow; this is a side effect of the Last 10% Trap. Most companies end up knuckling under the framework, modifying their processes rather than trying to customize the tool. The more companies do that, the less differentiators exist between companies, which is fine as long as that differentiation isn’t a competitive advantage. Milano Apache Kafka® Meetup / quantyca.it
  • 26. To big to remove Milano Apache Kafka® Meetup / quantyca.it Having developed and optimized Legacy King Systems for decades, enterprises are looking for ways to capitalize on these investments with the ability to reuse, repurpose, and integrate them in support of new business and IT initiatives.
  • 27. Legacy offloading Milano Apache Kafka® Meetup / quantyca.it DB2/MF IMS VSAM SAP Legacy King Data Warehouse Straem Analytics Hybrid Cloud Microservices C-Level / Menagement Customer LoB Analyst WHERE DATA NEED TO BE KAFKA OFFLOADING
  • 28. Offloading via batch loading ● Queries based on timestamp orchestrated with custom ETL Jobs ● Consume significant processing power on the source system ● Usually executed during “batch windows” (not responsive) ● Unable to manage easily changes in the schema structure ● Custom error handling and failover management Milano Apache Kafka® Meetup / quantyca.it @see Streaming change data capture
  • 29. Offloading via CDC Milano Apache Kafka® Meetup / quantyca.it ● Read incremental changes directly from the transaction log ● Minimize disruption to production workloads ● Fast updates (near real-time) ● Manage easily changes in the schema structure ● Mature market @see Streaming change data capture
  • 30. Agenda Milano Apache Kafka® Meetup / quantyca.it 4 First steps in the evolutionary journey 3 The destination: message driven architecture 5 Legacy offloading 2 The starting point: spaghetti architecture 1 Why we need to evolve 6 Attunity replicate and the CDC maturity model
  • 31. Attunity Replicate Milano Apache Kafka® Meetup / quantyca.it @see Attunity Replicate Home Page
  • 32. Attunity: workflow Milano Apache Kafka® Meetup / quantyca.it STEP 1: Create connections STEP 2: Create task STEP 3: Define task
  • 33. Attunity: workflow Milano Apache Kafka® Meetup / quantyca.it STEP 4: Run task
  • 34. Attunity: Source and targets Milano Apache Kafka® Meetup / quantyca.it
  • 35. Attunity: Kafka target Milano Apache Kafka® Meetup / quantyca.it
  • 36. Attunity: business cases Milano Apache Kafka® Meetup / quantyca.it
  • 37. Offloading maturity model Milano Apache Kafka® Meetup / quantyca.it Data offloading Process offloading Business offloading
  • 38. Thanks! Questions? Milano Apache Kafka® Meetup / quantyca.it Milano, 4.07.2018
  • 39. Corso Milano, 45 / 20900 Monza (MB) T. +39 039 9000 210 / F. +39 039 9000 211 / @ [email protected] www.quantyca.it

Editor's Notes

  • #12: https://www.reactivemanifesto.org/
  • #14: In a microservices architecture, each bounded context represents a business process or workflow.
  • #15: In a microservices architecture, each bounded context represents a business process or workflow.
  • #16: The Continuous Delivery and DevOps movements added a new factor into the dynamic equilibrium. Now, machine definitions live in version control and support extreme automation. Deployment pipelines spin up multiple test environments in parallel to support safe continuous deployment. Because much of the software stack is open source, licensing and other concerns no longer impact architectures. The community reacted to the new capabilities emergent in the software development ecosystem to build more domain-centric architectural styles. Make sure your architecture matches the problem domain. Don’t try to force fit an unsuitable architecture.
  • #18: Larger service granularity The services in this architecture tend to be larger, more “portion of a monolith” granularity than purely around domain concepts. While they are still domain-centric, the larger size makes the unit of change (development, deployment, coupling, and a host of other factors) larger, diminishing the ability to make change easily. When architects evaluate a monolithic application, they often see coarse-grained divisions around common domain concepts such as CatalogCheckout or Shippping, which form a good first-pass at partitioning the architecture. The goals of operational isolation are the same in service-based architectures as in microservices but are more difficult to achieve. Because the service size is bigger, developers must consider more coupling points and the complications inherent in larger chunks of code. Ideally, the architecture should support the same kind of deployment pipeline and small unit of change as microservices: when a developer changes a service, it should trigger the deployment pipeline to rebuild the dependent services, including the application. Database scope Service-based architectures tend towards a monolithic database, regardless of how well-factored the services are. In many applications, it isn’t feasible or possible to restructure years (or decades) of intractable database schemas into atomic-sized chunks for microservices. While the inability to atomize the data may be inconvenient in some situations, it is impossible in some problem domains. Heavily transactional systems are a poor match for microservices because coordination between services, transactional behavior is too costly. Systems with complex transactional requirements map more cleanly to service-based architectures because of less stringent database requirements. While the database remains unpartitioned, the components that rely on the database will likely change, becoming more granular. Thus, while the mapping between the services and the underlying data may change, it requires less restructuring. We cover evolutionary database design in Chapter 5. Integration middleware The third difference between microservices and service-based architectures concerns externalized coordination via a mediator like a service bus. Building greenfield microservices applications allows developers to not worry about old integration points, but those horrors describe many environments rife with legacy systems that still perform useful work. Integration hubs, like enterprise service buses, excel at forming glue between disparate services with different protocols and message formats. If architects find themselves in environments where integration architecture is the top priority, using an integration hub makes adding and changing dependent services easier.
  • #26: Rather than fall victim to the vendor king antipattern, treat vendor products as just another integration point. Developers can insulate vendor tool changes from impacting their architecture by building anticorruption layers between integration points.