MuCon 2015 - Microservices in Integration Architecture
13 likes5,874 views
The document discusses integration architecture in a microservices world. It begins by defining integration architecture as how data and functions are shared between applications. It then discusses challenges with large enterprise landscapes that have undergone mergers and acquisitions. The document outlines different types of integration architectures like external, enterprise, batch-based, and event-based integration. It also discusses common misconceptions around microservices, such as thinking microservices refer to exposed APIs rather than application components. The summary concludes by noting debates around the differences between microservices and service-oriented architecture (SOA).
MuCon 2015 - Microservices in Integration Architecture
- 1. Where is Integration in a Microservices world? Kim Clark 9th October MuCon 2015
- 2. What do we mean by “Integration Architecture”? How data and function are shared between applications within and across enterprise boundaries 1 Application Application Application Application Application Application Point to point Hub and spoke Service Oriented ArchitectureEnterprise Service Bus Messaging APIs Managed File Transfer Business to Business (B2B) Extract Transform Load Connectors / Adapters Mapping / Transformation Enterprise Application Integration Application Microservice application µService µService µService µService
- 3. Large enterprise landscapes are horrible! • Mergers and acquisitions • Multiple systems of record for the same data types • Non-correlating duplicates • Account centric vs. Customer centric • Version locked systems • There are always 3 versions of a system – The one you’re migrating from – The one you’re on now – The one you’re migrating too • Months to add a fix pack • Very long lived entities • Mortgages, deeds, contracts, pensions, insurance policies • Evolved complexity • Decades of business logic embedded alongside the data • Business Partner interfaces • Exceptions are the rule • Too important to deprecate 2
- 4. Integration architecture subsets External (API) Exposure Enterprise (Service/API) Exposure Batch-based Hub and Spoke Batch Transfer Direct Batch Transfer B2B Event-based Hub and Spoke Event-based Point to Point Real-time Direct Access External Event Management LocalEnterprisePublic/Partner Batch-based Event-based Connected
- 5. What does a large scale integration landscape look like Systems of record • Older technology • Harder to change • Challenging integration points Integration Hub Integration Hub Adapter Adapter “Systems of Engagement” Applications Exposure Gateway (internal) “Systems of Record” Applications Mature large enterprise (simplified) Exposure Gateway (external) Adapter Adapter “Hub and Spoke” integration “Enterprise Service Bus?” “Web API Gateway” Systems of engagement • Modern languages/runtimes • Agile updates • Simple modern connectivity
- 6. Confusions that arise around microservices 5
- 7. Obligatory “What is a microservice” slide Monolithic application Microservices application Silo Microservice (component) Microservice (component) Microservice (component) Agility Scalability Resilience
- 8. Microservice component Common misconception resulting from the term “microservice” Monolithic application Microservices application Exposed services/APIs Microservice component Microservice component Exposed services/APIs Silo component Microservices are fine grained WSDL operations APIs are microservices The “service” in “microservice” refers to the granularity of the components, not the granularity of the exposed interfaces x 1 x 3 x 4x 4 Is “microservices architecture” is really “micro-component architecture”?
- 9. Application SOA relates to enterprise service exposure Application ApplicationApplication Service oriented architecture (SOA) and microservices architecture relate to different scopes Microservice application µService µService µService µService Microservices relate to application architecture However, even this simple distinction can be contentious depending on your definition of SOA
- 10. What was SOA really about? Integration or Components? “SOA is about how to achieve integration often to aging complex back end systems in order to expose services” In this case SOA is primarily a connectivity problem with little relationship to microservices architecture and certainly at a different scope. “SOA is about re-factoring your IT landscape into components that better align with the business needs and expose the services that it needs” Here the connectivity problem is pushed down to the applications and the focus of SOA is on realignment of to the business needs. The service components look more like applications, and we might consider microservices as “more granular SOA”, or even “SOA done right”. System of Record Integration Hub Adapter Adapter Exposure Gateway Exposed Services/APIs System of Record System of Record Adapter Exposed Services/APIs System of Record Adapter Service Component Exposure Gateway 9 Service Component Service Component
- 11. Why such split opinions on microservices vs SOA? 10 Integration Hub Integration Hub Adapter Adapter Engagement Applications SaaS Application BusinessPartner Systemsof Record Business Partner Exposure Gateway (external) Mature large enterprise Microservices are just one style of application Exposing services is an integration and data challenge Green field online start-up Much of landscape could be microservice based The landscape is as (micro)service oriented architecture Exposure Gateway (external) Microservice application µService µServiceµService µService Exposure Gateway (internal) µService µService µService µService µService µService µService µService µService µService µService EnterpriseBoundary EnterpriseBoundary Adapter Adapter
- 12. SOA principles get a second chance with microservices 11 Challenges of SOA era Advantages in microservices era Funding Poor funding model. No project wanted to pay for re- usable services. API Economy provides potential funding mechanisms. Costs of integration reducing. Integration Connectivity was hard. Standards were immature. Tools were just emerging. Integration standards and tools improved. Modern APIs often provided as out of the box. Data Data landscape disjoined. Duplication common. Function embedded with data. SOA/MDM initiatives rationalised some landscapes. New data techniques to handle big data more flexibly. Requirements Hard to guess the perfect service model. Business needs constantly changing. Microservices lend themselves to lightweight agile techniques enabling more iterative requirements discovery. Result Integration led services. Poor re-use. Business led API requirements. Creation on demand.
- 13. Levels of API/service exposure Just because it’s REST/HTTP, doesn’t mean it’s all the same • How different is the APIs exposure style at each level? • How strong should the application boundaries be? 12 Microservices application Microservice Microservice Within-application Enterprise-wide Internet exposed API/service exposure scope
- 14. Microservice MicroserviceMicroservice Microservice Microservices inter-communication Aim is decoupling for robustness Messaging where possible • Lightweight messaging (e.g. AMQP, Kafka) • Publish/subscribe • Eventual consistency Direct calls where necessary Lightweight protocols (e.g. JSON/HTTP) • Load balancing/scaling via service discovery • Circuit breaker • Caching Microservices application Microservice Subscribe JSON /HTTP Microservice Microservice Message Hub API Microservice Publish API JSON /HTTP Publish Service Discovery
- 15. Application Internal vs. External/Partner exposure – comparing at the level of functionality/capability Firewall Canonical Integration Data Translation Routing/Distribution Composition Exposure Traffic Management - Basic Security - Basic Decoupling Visibility - Basic Partner Management Account based access Accounting/Billing Self Administration Service level management Discovery - Basic Exposure Traffic Management Security Decoupling Visibility Discovery API/Service Gateway API/Service Gateway Canonical Integration Data Translation - Basic Routing/Distribution - Basic Composition - Basic There may be only a handful of well understood internal consumer applications There may 10s of external consumer applications …and there could be hundreds of “experimenters”
- 16. Do todays systems of engagement become tomorrows systems of record? Engagement Application Logic Systemsof Record Presentation Data Logic Presentation Data Logic 1 2 3 1 New engagement application • Provides new, engaging, agile presentation tier • Cache/subset of data, for temporary and/or low latency storage • Integrates with enterprise services/APIs easily. 2 Aging engagement application • Presentation less attractive and agile • Data model broadened, with increased integrity as now used by other applications • Struggles to integrate with other systems 3 Becomes system of record • Presentation tier no longer used. Logic/data now accessed and/or replicated to new systems of engagement. • Data model stabilised, and hard to change, with increased integrity as now used by other applications • Requires external tools to integrate with other systems. 1. How long before your microservice application becomes one of the “legacy” systems? 2. How long before you have more than one mircoservices platform? Exposure Gateway
- 17. Heterogeneous on the inside, homogeneous on the outside Freedom to choose runtimes, languages, datastores etc. may be a birthright with microservices, but don’t let it become a millstone. • Choose runtimes based on their properties, not the preferences/skills of the team • Look for natural convergence on runtimes, then embrace it. Seek to push maximum commonality into the framework surrounding the microservices: • Scaling • Workload distribution • Resilience • Service discovery • Deployment orchestration • Messaging • Logging • Testing 16 Microservices application Microservice (Node.js) Microservice (Java + MongoDB) Microservice (Go + CloudantDB)
- 18. But what is an application boundary in an microservices world? Similar to application boundaries today because • Group related functionality • Enable more coarse grained ownership and responsibility (in addition to full ownership at the microservice level) Different because • Application sub-components (microservices) are truly independent. • Application boundaries can be easily changed – it’s primarily about the right ownership • Application boundaries can spread across network boundaries (e.g. between clouds)…but should they! 17 SaaS Application Business Partner Exposure Gateway (external) µService µService µService µService µService µService µService µService µService µService µService EnterpriseBoundary
- 19. Traditional Engagement Application System of record System of record SaaS Application Decentralising service/API exposure In a perfect world… • Applications self-administer exposure of their own APIs • API monitoring/diagnostics would be gathered across the organisation • Security models could be implemented more consistently. • No additional team required to enable integration • Application logic is firmly seated with the application teams 18 Business Partner EnterpriseBoundary Public API Private API API Gateway Microservice Engagement Application
- 20. In reality, it will be a long haul for many organisations… 19 Systems ofRecord Integration Hub Integration Hub Adapter Adapter Engagement Applications Microservice applications SaaSApplications (external) Adapter Externally Exposed Services/APIs Exposure Gateway (internal) Exposure Gateway (external) BusinessPartners
- 21. Caution with comparisons (or “how to start an argument”!) Comparisons that make some level of sense Microservice architecture vs. Service oriented architecture Trying to achieve many of the same things (enable the creation of re-usable business functions as isolated components), but perhaps a different scale. SOA evolved to become focused on the enterprise scale (though some might say that wasn’t the original intent). Microservices, is primarily at the application scale. Microservice component vs. Service component Both describe the actual implementation of a business function. They differ primarily in granularity, but also arguably in maintainability, agility etc. Confusing comparisons Microservices vs. SOA Microservices are components. SOA is an architecture. What you’re probably trying to compare is Microservices Architecture with SOA (see above). Microservices vs. APIs APIs are a interface. A mechanism for exposing business function. Microservices are the component architecture used to implement those business functions. It makes no sense to directly compare them. Microservice vs. Service “Service” means different things to different people. Needs to be qualified. Service Implementation? Service exposure? Service definition? The same could be said for microservice. Need to be specific. 20