The Hardest Part of Microservices: Calling Your Services
Download as PPTX, PDF5 likes2,567 views
When building microservices, you must solve for a number of critical functions, but the process can be incredibly complex and expensive to maintain. Christian Posta offers an overview of Envoy Proxy and Istio.io Service Mesh, explaining how they solve application networking problems more elegantly by pushing these concerns down to the infrastructure layer and demonstrating how it all works.
Ad
More Related Content
What's hot (20)
Similar to The Hardest Part of Microservices: Calling Your Services (20)
Ad
More from Christian Posta (19)
Ad
Recently uploaded (20)
![Pixologic ZBrush Crack Plus Activation Key [Latest 2025] New Version](https://cdn.slidesharecdn.com/ss_thumbnails/fashionevolution2-250322112409-f76abaa7-250428124909-b51264ff-250504160528-fc2bb1c5-thumbnail.jpg?width=560&fit=bounds)
![Get & Download Wondershare Filmora Crack Latest [2025]](https://cdn.slidesharecdn.com/ss_thumbnails/revolutionizingresidentialwi-fi-250422112639-60fb726f-250429170801-59e1b240-thumbnail.jpg?width=560&fit=bounds)
The Hardest Part of Microservices: Calling Your Services
- 1. The hardest part of microservices: Calling your services Christian Posta (@christianposta) Chief Architect – Red Hat
- 2. Full slide deck here: http://bit.ly/ceposta-hardest-part
- 4. Christian Posta Chief Architect, cloud application development Twitter: @christianposta Blog: http://blog.christianposta.com Email: [email protected] Slides: http://slideshare.net/ceposta • Author “Microservices for Java developers” • Committer/contributor lots of open-source projects • Worked with large Microservices, web-scale, unicorn company • Blogger, speaker
- 10. @christianposta
- 11. @christianposta
- 12. The network… does what it wants. @christianposta
- 14. As we move to services architectures, we push the complexity to the space between our services. @christianposta
- 15. We need guiding “service principles” @christianposta
- 16. • Services shall be resilient when communicating • Failures should not jump boundaries (no cascading failures) • New releases should not impact production • Mean time to recover should approach zero • Security as a first-class citizen • Policies to eliminate unexpected usage Service principles
- 17. Have we had to solve for this in the past? @christianposta
- 21. • Highly centralized == centralized governance • Inadvertently scattered business logic away from business apps/services • Scalability issues? • Not ideal for use in cloud environments Some drawbacks to this approach?
- 22. • Netflix Hystrix (circuit breaking / bulk heading) • Netflix Zuul (edge router) • Netflix Ribbon (client-side service discovery / load balance) • Netflix Eureka (service discovery registry) • Brave / Zipkin (tracing) • Netflix spectator / atlas (metrics) “Microservices” patterns
- 24. Now we have “service hurdles” @christianposta
- 25. • Require specific language to bring in new services • A single language doesn’t fit for all use cases • How do you patch/upgrade/manage lifecycle? • Need strict control over application library choices Some drawbacks to this approach?
- 26. But I’m using Spring! • spring-cloud-netflix-hystrix • spring-cloud-netflix-zuul • spring-cloud-netflix-eureka-client • spring-cloud-netflix-ribbon • spring-cloud-netflix-atlas • spring-cloud-netflix-spectator • spring-cloud-netflix-hystrix-stream • ….. • ...... • @Enable....150differentThings
- 27. But I’m using Vert.x! • vertx-circuit-breaker • vertx-service-discovery • vertx-dropwizard-metrics • vertx-zipkin? • ….. • ...... @christianposta
- 28. Screw Java - I’m using NodeJS! JavaScript is for rookies, I use Go! But python is so pretty! I prefer unreadability… Perl for me! @christianposta
- 29. Things you must solve for because… distributed systems • Service discovery • Retries • Timeouts • Load balancing • Rate limiting • Thread bulk heading • Circuit breaking @christianposta
- 30. …continued • Routing between services (adaptive, zone-aware) • Deadlines • Back pressure • Outlier detection • Health checking • Traffic shaping • Request shadowing @christianposta
- 31. …continued • Edge/DMZ routing • Surgical / fine / per-request routing • A/B rollout • Internal releases / dark launches • Fault injection • Stats, metric, collection • Logging • Tracing
- 32. An implementation of “service mess” @christianposta Now we have …
- 33. Now we have … • 30 different libraries for each of 5 languages, and each with 3 frameworks • How do we maintain, upgrade, retire • classpath/namespace pollution • increases operational complexity • force specific languages • inconsistency • correctness
- 34. These are all horizontal concerns and apply to all services regardless of implementation. @christianposta
- 35. Let’s abstract this functionality to a single binary and apply to all services. • Allow heterogeneous architectures • Remove application-specific implementations of this functionality • Consistently enforce these properties • Correctly enforce these properties • Opt-in as well as safety nets @christianposta
- 36. @christianposta
- 37. Evolution of application networking
- 39. Envoy is… • service proxy • written in C++, highly parallel, non-blocking • L3/4 network filter • out of the box L7 filters • HTTP 2, including gRPC • baked in service discovery/health checking • advanced load balancing • stats, metrics, tracing • dynamic configuration through xDS
- 40. @christianposta
- 41. Envoy implements • zone aware, least request load balancing • circuit breaking • outlier detection • retries, retry policies • timeout (including budgets) • traffic shadowing • rate limiting • access logging, statistics collection • Many other features!
- 42. As an edge proxy
- 43. As an shared proxy
- 44. As a service-instance proxy
- 45. Service instance proxy AKA Sidecar
- 47. Service mesh
- 48. “2018 is the year of the service mesh” Clayton Coleman (@smarterclayton) Red Hat OpenShift Platform Architect @christianposta
- 49. How do we reason about a fleet of these service proxies in a large cluster? @christianposta
- 50. A service mesh is decentralized application- networking infrastructure between your services that provides resiliency, security, observability, and routing control. A service mesh is comprised of a data plane and control plane. @christianposta Time for definitions:
- 51. All traffic between our applications flows through these proxies. The proxies make up the “data plane” @christianposta
- 52. Meet Istio.io http://istio.io A control plane for service proxies
- 55. What higher-order clusters semantics does Istio enable? • Request-level control • Graduated deployment and release • Service observability • Cluster reliability • Chaos testing • Policy enforcement
- 58. Thanks! BTW: Hand drawn diagrams made with Paper by FiftyThree.com Twitter: @christianposta Blog: http://blog.christianposta.com Email: [email protected] Slides: http://slideshare.net/cepostaFollow up links: • http://envoyproxy.io • http://istio.io • http://blog.christianposta.com/istio-workshop/slides/ • http://launch.openshift.io • http://blog.openshift.com • http://developers.redhat.com/blog • https://www.redhat.com/en/open-innovation-labs
Editor's Notes
- #6: One large database! We should focus on how we design our data models so that they can be sharded and distributed…. Focus on transactions, etc not 2PC
- #7: One large database! We should focus on how we design our data models so that they can be sharded and distributed…. Focus on transactions, etc not 2PC
- #8: One large database! We should focus on how we design our data models so that they can be sharded and distributed…. Focus on transactions, etc not 2PC
- #9: One large database! We should focus on how we design our data models so that they can be sharded and distributed…. Focus on transactions, etc not 2PC
- #10: One large database! We should focus on how we design our data models so that they can be sharded and distributed…. Focus on transactions, etc not 2PC
- #11: One large database! We should focus on how we design our data models so that they can be sharded and distributed…. Focus on transactions, etc not 2PC
- #12: One large database! We should focus on how we design our data models so that they can be sharded and distributed…. Focus on transactions, etc not 2PC
- #13: https://en.wikipedia.org/wiki/Fallacies_of_distributed_computing The network is reliable. Latency is zero. Bandwidth is infinite. The network is secure. Topology doesn't change. There is one administrator. Transport cost is zero. The network is homogeneous.
- #27: This concept of defining language, developing models to describe a domain, implementing those models, enforcing assertions, etc all happen within a certain context, and that context is vitally important in software. In common language, we are smart enough to resolve these types of language conflicts within a sentence because of its context. The computer doesn’t have this context. We have to make it explicit. And any context needs to have explicit boundaries. This model needs to be “useful” ie, it should be able to be implemented. Try to establish a model that’s both useful for discussion with the domain experts and is implementable. There are infinite ways to model/think about something. Balance both masters with the model you choose. Large complex domains may need multiple models. And really the only way to understand a language and model is within a certain context. That context should have boundaries so it doesn’t bleed or force others to bleed definitions and semantics. Bounded context: within this space, this is the context of the language. This is what it means and it’s not ambiguous. Central thing about a model is the language you create to express the prblem and solution very crisply. Need clear language and need boundaries. Anti corruption layers are translations between the different models that may exist in multiple bounded contexts. They keep an internal model consistent and pure without bleeding across the boundaries. Bounded contexts tend to be “self contained systems” themselves with a complete vertical stack of the software including UI, business logic, data models, and database. They tend to not share databases across multiple models.
- #28: This concept of defining language, developing models to describe a domain, implementing those models, enforcing assertions, etc all happen within a certain context, and that context is vitally important in software. In common language, we are smart enough to resolve these types of language conflicts within a sentence because of its context. The computer doesn’t have this context. We have to make it explicit. And any context needs to have explicit boundaries. This model needs to be “useful” ie, it should be able to be implemented. Try to establish a model that’s both useful for discussion with the domain experts and is implementable. There are infinite ways to model/think about something. Balance both masters with the model you choose. Large complex domains may need multiple models. And really the only way to understand a language and model is within a certain context. That context should have boundaries so it doesn’t bleed or force others to bleed definitions and semantics. Bounded context: within this space, this is the context of the language. This is what it means and it’s not ambiguous. Central thing about a model is the language you create to express the prblem and solution very crisply. Need clear language and need boundaries. Anti corruption layers are translations between the different models that may exist in multiple bounded contexts. They keep an internal model consistent and pure without bleeding across the boundaries. Bounded contexts tend to be “self contained systems” themselves with a complete vertical stack of the software including UI, business logic, data models, and database. They tend to not share databases across multiple models.
- #29: Get back to first principles. Focus on principles, patterns, methodologies. Tools will help, but you cannot start with tools.
- #30: This concept of defining language, developing models to describe a domain, implementing those models, enforcing assertions, etc all happen within a certain context, and that context is vitally important in software. In common language, we are smart enough to resolve these types of language conflicts within a sentence because of its context. The computer doesn’t have this context. We have to make it explicit. And any context needs to have explicit boundaries. This model needs to be “useful” ie, it should be able to be implemented. Try to establish a model that’s both useful for discussion with the domain experts and is implementable. There are infinite ways to model/think about something. Balance both masters with the model you choose. Large complex domains may need multiple models. And really the only way to understand a language and model is within a certain context. That context should have boundaries so it doesn’t bleed or force others to bleed definitions and semantics. Bounded context: within this space, this is the context of the language. This is what it means and it’s not ambiguous. Central thing about a model is the language you create to express the prblem and solution very crisply. Need clear language and need boundaries. Anti corruption layers are translations between the different models that may exist in multiple bounded contexts. They keep an internal model consistent and pure without bleeding across the boundaries. Bounded contexts tend to be “self contained systems” themselves with a complete vertical stack of the software including UI, business logic, data models, and database. They tend to not share databases across multiple models.
- #31: This concept of defining language, developing models to describe a domain, implementing those models, enforcing assertions, etc all happen within a certain context, and that context is vitally important in software. In common language, we are smart enough to resolve these types of language conflicts within a sentence because of its context. The computer doesn’t have this context. We have to make it explicit. And any context needs to have explicit boundaries. This model needs to be “useful” ie, it should be able to be implemented. Try to establish a model that’s both useful for discussion with the domain experts and is implementable. There are infinite ways to model/think about something. Balance both masters with the model you choose. Large complex domains may need multiple models. And really the only way to understand a language and model is within a certain context. That context should have boundaries so it doesn’t bleed or force others to bleed definitions and semantics. Bounded context: within this space, this is the context of the language. This is what it means and it’s not ambiguous. Central thing about a model is the language you create to express the prblem and solution very crisply. Need clear language and need boundaries. Anti corruption layers are translations between the different models that may exist in multiple bounded contexts. They keep an internal model consistent and pure without bleeding across the boundaries. Bounded contexts tend to be “self contained systems” themselves with a complete vertical stack of the software including UI, business logic, data models, and database. They tend to not share databases across multiple models.
- #32: This concept of defining language, developing models to describe a domain, implementing those models, enforcing assertions, etc all happen within a certain context, and that context is vitally important in software. In common language, we are smart enough to resolve these types of language conflicts within a sentence because of its context. The computer doesn’t have this context. We have to make it explicit. And any context needs to have explicit boundaries. This model needs to be “useful” ie, it should be able to be implemented. Try to establish a model that’s both useful for discussion with the domain experts and is implementable. There are infinite ways to model/think about something. Balance both masters with the model you choose. Large complex domains may need multiple models. And really the only way to understand a language and model is within a certain context. That context should have boundaries so it doesn’t bleed or force others to bleed definitions and semantics. Bounded context: within this space, this is the context of the language. This is what it means and it’s not ambiguous. Central thing about a model is the language you create to express the prblem and solution very crisply. Need clear language and need boundaries. Anti corruption layers are translations between the different models that may exist in multiple bounded contexts. They keep an internal model consistent and pure without bleeding across the boundaries. Bounded contexts tend to be “self contained systems” themselves with a complete vertical stack of the software including UI, business logic, data models, and database. They tend to not share databases across multiple models.
- #35: Get back to first principles. Focus on principles, patterns, methodologies. Tools will help, but you cannot start with tools.
- #36: Get back to first principles. Focus on principles, patterns, methodologies. Tools will help, but you cannot start with tools.
- #37: Get back to first principles. Focus on principles, patterns, methodologies. Tools will help, but you cannot start with tools.
- #39: Get back to first principles. Focus on principles, patterns, methodologies. Tools will help, but you cannot start with tools.
- #40: Get back to first principles. Focus on principles, patterns, methodologies. Tools will help, but you cannot start with tools.
- #41: Get back to first principles. Focus on principles, patterns, methodologies. Tools will help, but you cannot start with tools.
- #42: Get back to first principles. Focus on principles, patterns, methodologies. Tools will help, but you cannot start with tools.
- #43: Get back to first principles. Focus on principles, patterns, methodologies. Tools will help, but you cannot start with tools.
- #44: Get back to first principles. Focus on principles, patterns, methodologies. Tools will help, but you cannot start with tools.
- #45: Get back to first principles. Focus on principles, patterns, methodologies. Tools will help, but you cannot start with tools.
- #46: Get back to first principles. Focus on principles, patterns, methodologies. Tools will help, but you cannot start with tools.
- #47: Get back to first principles. Focus on principles, patterns, methodologies. Tools will help, but you cannot start with tools.
- #49: Get back to first principles. Focus on principles, patterns, methodologies. Tools will help, but you cannot start with tools.
- #50: Get back to first principles. Focus on principles, patterns, methodologies. Tools will help, but you cannot start with tools.
- #51: Get back to first principles. Focus on principles, patterns, methodologies. Tools will help, but you cannot start with tools.
- #52: Get back to first principles. Focus on principles, patterns, methodologies. Tools will help, but you cannot start with tools.
- #53: Get back to first principles. Focus on principles, patterns, methodologies. Tools will help, but you cannot start with tools.
- #54: Get back to first principles. Focus on principles, patterns, methodologies. Tools will help, but you cannot start with tools.
- #55: Get back to first principles. Focus on principles, patterns, methodologies. Tools will help, but you cannot start with tools.
- #56: Get back to first principles. Focus on principles, patterns, methodologies. Tools will help, but you cannot start with tools.
- #57: Get back to first principles. Focus on principles, patterns, methodologies. Tools will help, but you cannot start with tools.
- #58: Get back to first principles. Focus on principles, patterns, methodologies. Tools will help, but you cannot start with tools.