Manging Container Deployments at Scale

Editor's Notes

• #4: MR - Lets talk about why we are really here today. Bookinfo app. This is the next best thing after sliced bread. -> next
• #6: MR – So Jesse, should we build a nice little monolith for our app? I mean it will be the easiest to build. JB – But what if we have to change and scale the app differently for different functionality. And who are we kidding. This app will be so big, soon we will have to hire a full team to manage this.
• #7: MR – That makes sense. In a monolith all the apps will talk to same db and any change will mean we change the whole deployment. JB – Also adding new features is not always easy. More over you are kind of stuck to the same language stack. I know Java and Ruby. But you probably want to write the app with something you feel more comfortable with. MR – Right tool for the job amirite.
• #8: Audience check
• #9: JB – Lets build this app using microservices instead. You can write your application based on functionality. We can then Connect it all up using http protocol.
• #10: MR – That makes sense. We decouple our functionalities so we can build and grow our applications individually. Nothing looks different from the point of view of our user. But makes it much easier to make a change or improvement with out breaking the entire application.
• #11: JB – I took the liberty to design out the application while you were talking. We will have 4 microservices. MR – Why do we have 3 different reviews app. JB – Well our intensive market research shows some people may not like the stars at the review. Others has issues with the color of the stars. So we will experiment with the different colors for now.
• #12: MR – Btw, we open sourced this code for all to take and learn from
• #13: MR – So we are done. App is written. We just get these to run. And JB – Not so fast. We actually have to go through dev ops first.
• #14: MR – We have our application now. But since they are all independently developed and deployable. Devops is a whole another issue. JB – I ran the code in my computer and it worked. But bob from ops team said something is wrong with the deployment. MR – BUT DID you tell him though. JB – Tell him what?
• #16: Microservices – iterative development, rapid release, super fact pivot and time to market CI / CD - if you’re going to release fast, you need a system to keep up with it Cloud adoption - while not strictly required, at this point… yeah. Containers. - just as with microservice architecture and ci / cd – this was a good fit for the methodology these technologies became prolific and ubiquitous because they were needed by the methodology, we did not adapt the methodology to fit the tech
• #17: JB – We can now dockerize our app in a container that has all the dependency it needs to run our application. Any place that has a docker engine now can run the app the same way for the most part. MR – Docker changed the game in many way. While not a new technology. My Linux diehards know the tech to isolate namespace in linux have been around forever. But docker lets us use easy apis and tooling to makes this process much more streamlined.
• #18: MR – So jesse, I dockerized all our microservices. JB – How does one dockerize and APP.
• #19: MR – Using this artifacts called dockerfiles of course. I can define everything I need to run my app. And docker builds it for me. Then I can push these of to a registry. JB – Hah, too slow. I already did.
• #20: JB – All we have to do now. Is run these docker images. MR – Nice. You know what this means?
• #22: MR – So our app is ready. Lets run it. JB - Well one container went down. No big deal. I can spin up more. MR – Wait more stuff is going down. And I cant really check why. Spin up more. JB – I don’t know whats happening. Its all on fire. MR – we really need to have a way to manage this containers. I wish there was a way that the platform could handle auto recovery from failure and scaling.
• #23: JB – There might just be a way.
• #24: MR – Kubernetes to the rescue.
• #25: JB - Kubernetes provides abstractions for deploying software in containers at scale Again, out of necessity - containers were everywhere, and various orchestration options arose. Mesos, Docker Swarm, others… Kubernetes won
• #26: IMR - nfrastrcture resources are abstracted in a cluster of worker nodes, and the cluster has a scheduler which deploys work to those nodes So, everything we need… JB – I know we have spent countless hours in making our app dockerized. How do we now get it to run on kubernetes.
• #27: MR – Its not that bad. We kubernetize our app (you heard it here first folks, I am trying to get this word trending, some one tweet #kubernetize ) JB – I see you are using the image we built and pushed up earlier. MR – Yeah and we are opening up the same port for communication. JB – <Talks about deployment> MR - <Talks about service>
• #28: JB - so now rather than a monolithic application, running in bespoke compute environments
• #29: JB - this is day one. (-> next)
• #30: JB - what do we pick up on day two? or put another way, what happens when we succeed, and our prototyped happy path software needs to scale? JB – But in either way, our app is running, and we are ready to do anything. MR – You know what this mean.
• #32: JB – But wait, did not kelcey warn against this. MR – IDK Jesse our way seems pretty solid. Lets see where it goes.
• #33: JB – App is running great. We getting some traffic, but Kubernetes has autoscaling. MR – Yeah. And we don’t have time debugging a failed instance. Also kubernetes just spins up a new copy so we cant even look at the logs. JB – More stuff is going down. Why is that one service taking forever to respond. Wait that’s returning 500 now? MR – Wait didn’t we just have something like this with our docker setup.
• #34: JB – I don’t know Mofi. I think we just need to live with this. MR – Maybe there is something more out there.
• #35: MR – Back to the drawing board with this one. We need a way to simply and repeatably deploy, and simply and recoverably modify. Kubernetes has our backs there. JB - We need to re-establish telemetry, observability, and diagnosibility as table stakes for computing at scale. Here, we need to bring some stuff to the party. I want to benefit of microservices with the observability of a monolith.
• #36: MR – No worries. ingress and traffic managemnet, we can use nginx for that. JB - tracing and observabilty, No sweat, open tracing to the rescue. MR - metrics and analytics. Prometheus all day. JB –I know we are forgetting something. Uhh. Wait security. We never thought about security. I think we can use vault for that.
• #37: MR – so it kind of sounds like what we want is something that gives us these things. MR – These looks pretty hard to get.
• #38: JB – Hard things are hard mofi. JB - …
• #39: MR – Well I know, we will use a service mesh. I it in a talk once. JB – Whats with you and throwing around big words randomly like that?
• #40: MR - The term service mesh is used to describe the network of microservices that make up such applications and the interactions between them. 
• #41: JB - A service mesh is an ideal component in a DevOps environment, as it provides operators with a stable and extensible platform for all of the work needed to maintain and improve the platform, while it remains completely invisible to developers
• #42: MR - Service Mesh This is not a new solution which solves all the world's problems It allows for integration of all existing (and future) best in class solutions for All the Things JB - First time I’ve used that gif unsarcastically So, how does it do that? For that…
• #43: MR - Istio a service mesh that allows us to connect, secure, control and observe services at scale, often requiring no service source code modification
• #44: Envoy we’ve talked about this, the proxy pilot converts high level routing rules that control traffic behavior into Envoy-specific configurations, and propagates them to the sidecars at runtime Mixer Mixer enforces access control and usage policies across the service mesh, and collects telemetry data from the Envoy proxy and other services citadel provides strong service-to-service and end-user authentication with built-in identity and credential management.
• #45: JB - fine-grained traffic control with rich routing rules, retries, failovers, and fault injection Automatic metrics, logs, and traces for all traffic within a cluster, including cluster ingress and egress
• #46: MR - security - a strong identity-based authentication and authorization layer which is secure by default for ingress, egress and service-to-service traffic policy - layered over all of this is a pluggable policy engine supporting access controls, rate limits and quotas JB - typically we get all of this with little to no application code changes MR – I don’t believe you.
• #47: MR – Yeah I am with skeptical kid on this one. How would that even work?
• #48: What does this look like: two services talking to each other – HTTP GET, simple
• #49: JB - IP Tables rules are automated to intercept all service traffic and reroute to proxy MR - The proxy has rules and policies to follow, and after considering policy, routing rules, etc, it forwards the traffic to the appropriate service Let’s talk about that little proxy box
• #50: MR – So Jesse things this is the work of the Hackermant
• #51: JB – and mofi things this is just magic
• #52: MR – But in reality this is all done at the envoy level. JB - Definitely could be its own talk, and there are many out there to check out. Envoy is a good example of Istio surfacing other features of a best-in-class component through its mesh .
• #53: Ok so, back to this
• #54: HTTP HTTP/2 supported, gRPC, or anything over TCP… with or w/o mTLs
• #55: pilot managing the proxies Mixer handling enfocement and telemtrey pickup citadel authN authZ
• #57: Differentiate API gateway – primaritly north-south, vs service mesh east-west Istio has gateway’s which provide ingress for the mesh Betyond that, a lot of day-to-day becomes really simple – canary, traffic mixing for blue/green, AB testing
• #59: We get most of this for free
• #60: More on testing Integrated benchmarking virtually free, making it incredibly easy to catch version-to-version regression
• #61: And all of this is safe out of the box, secure by default in depth with multiple components