Kubernetes & Redis: Aparna Sinha

Editor's Notes

• #2: Good morning everybody! I’m really excited to talk to you about two of the most popular technologies in use today: Redis and Kubernetes.
• #3: You all know that Redis was voted the most loved database on Stack Overflow surveys consecutively for the last two years, and it is ranked the most popular key-value database by DB-Engines and others. Kubernetes is not looking too shabby either. Some are calling it the ‘Linux of the Cloud’, as a sort of operating system for distributed computing. At Google we’ve used predecessors of this technology for decades so it is natural
• #4: But externally too Kubernetes is seeing strong momentum. It’s in the top 0.01% of GitHub projects, 54% of Fortune 100 companies appear to be using it, and jobs, training and contributions are all growing.
• #5: To be honest, developers don’t really choose their tools based on statistics. So I asked GKE users why do they use Kubernetes, and they tell me it does a few things unexpectedly well. #1. Kubernetes improves resource utilization, through efficient bin packing and auto-scaling. This saves you money. In fact on GKE, Google’s hosted Kubernetes offering, the majority of large clusters use cluster autoscaling saving upto 40% in costs over static placement. And we have recently added new dimensions and greater automation. #2. Kubernetes increases reliability. This saves your sanity. Again through a combination of monitoring, remediation, and a host of security features. On GKE for example we provide auto-repair and SRE monitoring for a more secure and reliable experience. #3. Kubernetes improves developer productivity. Developers are able to release many times a day. This is a top line benefit. Lastly it provides a consistent environment for running applications across any infrastructure
• #6: Kubernetes does these things so well that people want to run more and more applications on Kubernetes across industries. Containers and GKE started with stateless web services and were found to be really good also for batch applications On GKE, we have seen rapid growth in a larger variety of applications. For example ML usage on GKE has exploded since we automated provisioning and scaling of GPUs And over 40% of clusters on GKE are stateful (slide 7). This includes Redis, Elastic, Postgres, MySQL, Cassandra and many other databases. Infact, Redis is the MOST commonly used data-store on K8S.
• #7: So why are people using Redis with their Kubernetes applications on Google Cloud? Well, both have a few things in common Speed Portability Versatility
• #8: Speed Both Redis and Kubernetes are cloud native, they enable fast scaling and rapid iteration Developers deploy 100s of times a day with Kubernetes and you need a datastore that complements this WIth Redis you can change the way you use your data and its contents as fast as you change your application because data is in memory
• #9: Redis is portable because its a simple C application running on linux. Anywhere you turn up linux you can you can run Redis. Google is the open cloud. We have a long history of building and contributing to open source software. So its no surprise that Kubernetes is a management stack that can go anywhere containers can. The freedom to deploy apps on-prem, in hybrid or multi-cloud environments lets you choose based on the merits of the underlying cloud and also make changes as your business evolves. Perhaps the salient point is that Redis is frequently run on K8S e.g., on GKE. This lets users manage it easily and consistently anywhere.
• #10: Versatility: Redis [can build so much with it] Redis is known as the Swiss army knife of the internet: messaging, caching, queuing.
• #11: Versatility: Today Kubernetes is used across industries and for a variety of applications eBay for shopbot and Google Assistant NYT for ML article recommendations Philips HUE with Redis cache for IoT lights HSBC for mobile banking Also frequently for gaming and social media applications It is easy to see the overlap in types of applications that Kubernetes is used for and those that Redis is best for. Given this overlap it’s no wonder there is strong interest in making Redis easier to manage on Kubernetes.
• #12: Kubernetes extensibility is very much a conscious decision: We are moving away from built-in types to resource definitions that are extensible. They develop CRDs and controllers that are application specific those describe the resources the cluster needs to talk about the application. CR = declarative intent of what you want the infra to be. CRD = what’s permissible; the metadata for the CR Controller = you write to realize the declarative intent to realize the user intent The combination of a CR + CRD is called an operator. Kubernetes extensibility allows various applications to be run using Operators. Normally an ‘operator’ is a person who operates your application. They deploy, scale, upgrade and manage the app. Kubernetes Operators do the same thing. Now lets look at a Redis Operator and how it helps you.
• #13: What does Redis operator do? Turning up a Redis cluster takes time - start Redis, modify config to form cluster, deploy Sentinel, modify clients OR install Redis cluster and setup config to form a cluster. Won’t autoscale… On K8S scaling is trivial, process remediation is trivial. It does all the provisioning of compute, networking and storage for the cluster dynamically. If nodes / pods fail it keeps the infrastructure highly available. The operator stands up a Redis cluster quickly, without much manual effort.. Don’t have to provision infra or storage all based on declared intent. If I kill one, it just comes back. Attached to same storage and data is still there. Makes redis more production ready so you can use it for a broader set of use cases.
• #14: Let’s see a demo Here we have a Kubernetes cluster in Google cloud through our hosted service GKE I’m connected to it. Now I’m going to create a CRD Now I’m going to create the operator that uses the CRD Now any developer that wants to can create their own redis cluster using Kubectl on GKE Then I’ll create a redis cluster directly from Kubectl Now the redis cluster is being created.
• #15: As you saw, K8S makes Redis more reliable, scalable, and easier to manage, and because users are using Kubernetes for many different applications, I think we will all be doing more with Redis on K8S.
• #18: You all know that Redis was voted the most loved database on Stack Overflow surveys for consecutively for the last two years, and it is ranked the most popular key-value database by DB-Engines and others. Kubernetes is not looking too shabby either. (Which is partly why I’m here). Some are calling it the ‘Linux of the Cloud’.
• #19: But developers don’t really choose their tools based on statistics. Users tell me Kubernetes is useful because it does a few things unexpectedly well. #1. Kubernetes improves resource utilization, through efficient bin packing and auto-scaling. This saves you money. In fact on GKE, Google’s hosted Kubernetes offering, the majority of large clusters use cluster autoscaling saving upto 40% in costs over static placement. And we have recently added new dimensions and greater automation. #2. Kubernetes increase reliability. This saves your sanity. Again through a combination of monitoring, remediation, and a host of security features. On GKE for example #3. Kubernetes improves developer productivity. Developers are able to release many times a day. ANd this is a top line benefit. Lastly it provides a consistent environment for running applications across any infrastructure
• #20: Versatility: Today Kubernetes is used across industries and for a variety of applications NYT / Bloomberg for ML article recommendations Philips HUE with Redis cache for IoT lights eBay for shopbot and Google Assistant HSBC for mobile banking Also frequently for gaming and social media applications Niantic gaming Snap for social / elastic search It is easy to see the overlap in types of applications that Kubernetes is used for and those that Redis is best for. Given this overlap its no wonder there is strong interest in making Redis easier to manage on Kubernetes.
• #21: Versatility: Today Kubernetes is used across industries and for a variety of applications NYT / Bloomberg for ML article recommendations Philips HUE with Redis cache for IoT lights eBay for shopbot and Google Assistant HSBC for mobile banking Also frequently for gaming and social media applications Niantic gaming Snap for social / elastic search It is easy to see the overlap in types of applications that Kubernetes is used for and those that Redis is best for. Given this overlap its no wonder there is strong interest in making Redis easier to manage on Kubernetes.
• #22: What does Redis operator do? Turning up a Redis cluster takes time - start Redis, modify config to form cluster, deploy Sentinel, modify clients OR install Redis cluster and setup config to form a cluster. Won’t autoscale… On K8S scaling is trivial, process remediation is trivial. The operator stands up a Redis cluster quickly, without much manual effort.. Don’t have to provision infra or storage all based on declared intent. If I kill one, it just comes back. Attached to same storage and data is still there. Makes redis more production ready so you can use it for a broader set of use cases.