Ad
More Related Content
What's hot (20)
Similar to Architectural caching patterns for kubernetes (20)
![[jLove 2020] Where is my cache architectural patterns for caching microservi...](https://cdn.slidesharecdn.com/ss_thumbnails/whereismycachearchitecturalpatternsforcachingmicroservicesbyexample-201204105143-thumbnail.jpg?width=560&fit=bounds)
![[DevopsDays India 2019] Where is my cache? Architectural patterns for caching...](https://cdn.slidesharecdn.com/ss_thumbnails/slidesharewhereismycachearchitecturalpatternsforcachingmicroservicesbyexample-190913082107-thumbnail.jpg?width=560&fit=bounds)
Ad
More from Rafał Leszko (15)
Ad
Recently uploaded (20)
![Get & Download Wondershare Filmora Crack Latest [2025]](https://cdn.slidesharecdn.com/ss_thumbnails/revolutionizingresidentialwi-fi-250422112639-60fb726f-250429170801-59e1b240-thumbnail.jpg?width=560&fit=bounds)
Architectural caching patterns for kubernetes
- 1. Architectural Caching Patterns for Kubernetes Rafał Leszko @RafalLeszko rafalleszko.com Hazelcast
- 2. About me ● Cloud-Native Team Lead at Hazelcast ● Worked at Google and CERN ● Author of the book "Continuous Delivery with Docker and Jenkins" ● Trainer and conference speaker ● Live in Kraków, Poland
- 3. About Hazelcast ● Distributed Company ● Open Source Software ● 140+ Employees ● Products: ○ Hazelcast IMDG ○ Hazelcast Jet ○ Hazelcast Cloud @Hazelcast www.hazelcast.com
- 4. ● Introduction ● Caching Architectural Patterns ○ Embedded ○ Embedded Distributed ○ Client-Server ○ Cloud ○ Sidecar ○ Reverse Proxy ○ Reverse Proxy Sidecar ● Summary Agenda
- 5. Service 1 Service 2 v1 Service 2 v2 Service 1 Service 4 v1 Service 4 v2 Service 4 v3 Ruby Microservice World
- 6. Microservice World Service 1 Service 2 v1 Service 2 v2 Service 1 Service 4 v1 Service 4 v2 Service 4 v3 Ruby cache cache cache cache
- 7. Microservice World Service 1 Service 2 v1 Service 2 v2 Service 1 Service 4 v1 Service 4 v2 Service 4 v3 Ruby cache cache cache
- 8. Microservice World Service 1 Service 2 v1 Service 2 v2 Service 1 Service 4 v1 Service 4 v2 Service 4 v3 Ruby cache cache cache
- 9. ● Introduction ✔ ● Caching Architectural Patterns ○ Embedded ○ Embedded Distributed ○ Client-Server ○ Cloud ○ Sidecar ○ Reverse Proxy ○ Reverse Proxy Sidecar ● Summary Agenda
- 10. 1. Embedded
- 12. private ConcurrentHashMap<String, String> cache = new ConcurrentHashMap<>(); private String processRequest(String request) { if (cache.contains(request)) { return cache.get(request); } String response = process(request); cache.put(request, response); return response; } Embedded Cache (Pure Java implementation)
- 13. private ConcurrentHashMap<String, String> cache = new ConcurrentHashMap<>(); private String processRequest(String request) { if (cache.contains(request)) { return cache.get(request); } String response = process(request); cache.put(request, response); return response; } Embedded Cache (Pure Java implementation)
- 14. ● No Eviction Policies ● No Max Size Limit (OutOfMemoryError) ● No Statistics ● No built-in Cache Loaders ● No Expiration Time ● No Notification Mechanism Java Collection is not a Cache!
- 16. Embedded Cache (Java libraries) CacheBuilder.newBuilder() .initialCapacity(300) .expireAfterAccess(Duration.ofMinutes(10)) .maximumSize(1000) .build();
- 17. Caching Application Layer @Service public class BookService { @Cacheable("books") public String getBookNameByIsbn(String isbn) { return findBookInSlowSource(isbn); } }
- 18. Caching Application Layer @Service public class BookService { @Cacheable("books") public String getBookNameByIsbn(String isbn) { return findBookInSlowSource(isbn); } } Be Careful, Spring uses ConcurrentHashMap by default!
- 21. Application Application Kubernetes Service Cache Cache Request Hazelcast Cluster Embedded Distributed Cache Kubernetes Pod Kubernetes Pod
- 22. @Configuration public class HazelcastConfiguration { @Bean CacheManager cacheManager() { return new HazelcastCacheManager( Hazelcast.newHazelcastInstance()); } } Embedded Distributed Cache (Spring with Hazelcast)
- 25. Embedded Distributed Cache Application Application Kubernetes Service Cache Cache Request Hazelcast Cluster Kubernetes Pod Kubernetes Pod
- 26. Embedded Cache Pros Cons ● Simple configuration / deployment ● Low-latency data access ● No separate Ops Team needed ● Not flexible management (scaling, backup) ● Limited to JVM-based applications ● Data collocated with applications
- 27. ● Introduction ✔ ● Caching Architectural Patterns ○ Embedded ✔ ○ Embedded Distributed ✔ ○ Client-Server ○ Cloud ○ Sidecar ○ Reverse Proxy ○ Reverse Proxy Sidecar ● Summary Agenda
- 28. 2. Client-Server
- 29. Application Kubernetes Service Application Request Cache Server StatefulSet Client-Server Cache Kubernetes Pod Kubernetes Pod
- 30. Client-Server Cache Application Kubernetes Service Application Request Cache Server StatefulSet Kubernetes Pod Kubernetes Pod
- 31. Client-Server Cache Separate Management: ● backups ● (auto) scaling ● security Ops Team Application Kubernetes Service Application Request Cache Server StatefulSet Kubernetes Pod Kubernetes Pod
- 32. Client-Server Cache Application Kubernetes Service Application Request Cache Server StatefulSet Kubernetes Pod Kubernetes Pod
- 33. Client-Server Cache Application Kubernetes Service Application Request Cache Server StatefulSet Kubernetes Pod Kubernetes Pod
- 36. Client-Server Cache Starting Hazelcast Cache Server $ helm install hazelcast/hazelcast
- 37. Client-Server Cache Hazelcast Client @Configuration public class HazelcastClientConfiguration { @Bean CacheManager cacheManager() { ClientConfig clientConfig = new ClientConfig(); clientConfig.getNetworkConfig().getKubernetesConfig() .setEnabled(true); return new HazelcastCacheManager(HazelcastClient .newHazelcastClient(clientConfig)); } } Starting Hazelcast Cache Server $ helm install hazelcast/hazelcast
- 38. Client-Server Cache Separate Management: ● backups ● (auto) scaling ● security Ops Team Application Kubernetes Service Application Request Cache Server StatefulSet Kubernetes Pod Kubernetes Pod
- 39. 2*. Cloud
- 40. Application Kubernetes Service Application Request Cloud (Cache as a Service) Kubernetes Pod Kubernetes Pod
- 41. Cloud (Cache as a Service) Management: ● backups ● (auto) scaling ● security Ops Team Application Kubernetes Service Application Request Kubernetes Pod Kubernetes Pod
- 42. Cloud (Cache as a Service) Management: ● backups ● (auto) scaling ● security Ops Team Application Kubernetes Service Application Request Kubernetes Pod Kubernetes Pod
- 43. Cloud (Cache as a Service) Application Kubernetes Service Application Request Kubernetes Pod Kubernetes Pod
- 44. @Configuration public class HazelcastCloudConfiguration { @Bean CacheManager cacheManager() { ClientConfig clientConfig = new ClientConfig(); clientConfig.getNetworkConfig().getCloudConfig() .setEnabled(true) .setDiscoveryToken("KSXFDTi5HXPJGR0wRAjLgKe45tvEEhd"); clientConfig.setGroupConfig( new GroupConfig("test-cluster", "b2f984b5dd3314")); return new HazelcastCacheManager( HazelcastClient.newHazelcastClient(clientConfig)); } } Cloud (Cache as a Service)
- 45. Client-Server (Cloud) Cache Pros ● Data separate from applications ● Separate management (scaling, backup) ● Programming-language agnostic Cons ● Separate Ops effort ● Higher latency ● Server network requires adjustment (same region, same VPC)
- 46. ● Introduction ✔ ● Caching Architectural Patterns ○ Embedded ✔ ○ Embedded Distributed ✔ ○ Client-Server ✔ ○ Cloud ✔ ○ Sidecar ○ Reverse Proxy ○ Reverse Proxy Sidecar ● Summary Agenda
- 47. 3. Sidecar
- 48. Kubernetes Service Request Hazelcast Cluster Kubernetes POD Application Container Cache Container Application Container Cache Container Kubernetes POD Sidecar Cache
- 49. Sidecar Cache Similar to Embedded: ● the same physical machine ● the same resource pool ● scales up and down together ● no discovery needed (always localhost) Similar to Client-Server: ● different programming language ● uses cache client to connect ● clear isolation between app and cache
- 50. @Configuration public class HazelcastSidecarConfiguration { @Bean CacheManager cacheManager() { ClientConfig clientConfig = new ClientConfig(); clientConfig.getNetworkConfig() .addAddress("localhost:5701"); return new HazelcastCacheManager(HazelcastClient .newHazelcastClient(clientConfig)); } } Sidecar Cache
- 51. Sidecar Cache apiVersion: apps/v1 kind: Deployment ... spec: template: spec: containers: - name: application image: leszko/application - name: hazelcast image: hazelcast/hazelcast
- 52. Sidecar Cache Pros Cons ● Simple configuration ● Programming-language agnostic ● Low latency ● Some isolation of data and applications ● Not flexible management (scaling, backup) ● Data collocated with application PODs
- 53. ● Introduction ✔ ● Caching Architectural Patterns ○ Embedded ✔ ○ Embedded Distributed ✔ ○ Client-Server ✔ ○ Cloud ✔ ○ Sidecar ✔ ○ Reverse Proxy ○ Reverse Proxy Sidecar ● Summary Agenda
- 54. 4. Reverse Proxy
- 55. Application Kubernetes Service Cache Application Request Reverse Proxy Cache Kubernetes Pod Kubernetes Pod
- 56. Reverse Proxy Cache http { ... proxy_cache_path /data/nginx/cache keys_zone=one:10m; ... }
- 57. ● Only for HTTP ● Not distributed ● No High Availability ● Data stored on the disk NGINX Reverse Proxy Cache Issues
- 58. 4*. Reverse Proxy Sidecar
- 59. Kubernetes Service Request Hazelcast Cluster Kubernetes POD Application Container Reverse Proxy Cache Container Application Container Reverse Proxy Cache Container Kubernetes POD Reverse Proxy Sidecar Cache
- 61. Good
- 62. Reverse Proxy Sidecar Cache Service 1 Service 2 v1 Service 2 v2 Service 1 Service 4 v1 Service 4 v2 Service 4 v3 Ruby
- 63. Reverse Proxy Sidecar Cache Service 1 Service 2 v1 Service 2 v2 Service 1 Service 4 v1 Service 4 v2 Service 4 v3 Ruby
- 64. Reverse Proxy Sidecar Cache apiVersion: apps/v1 kind: Deployment ... spec: template: spec: initContainers: - name: init-networking image: leszko/init-networking containers: - name: caching-proxy image: leszko/caching-proxy - name: application image: leszko/application
- 65. Reverse Proxy Sidecar Cache Service 1 Service 2 v1 Service 2 v2 Service 1 Service 4 v1 Service 4 v2 Service 4 v3 Ruby
- 67. Reverse Proxy Sidecar Cache (Istio)
- 68. Reverse Proxy Sidecar Cache (Istio)
- 69. Bad
- 71. Reverse Proxy Cache @CacheEvict(value = "someValue", allEntries = true) public void evictAllCacheValues() {} Application Cache:
- 72. Reverse Proxy Cache @CacheEvict(value = "someValue", allEntries = true) public void evictAllCacheValues() {} Application Cache: Proxy Cache: http { ... location / { add_header Cache-Control public; expires 86400; etag on; } }
- 73. Reverse Proxy (Sidecar) Cache Pros Cons ● Configuration-based (no need to change applications) ● Programming-language agnostic ● Consistent with containers and microservice world ● Difficult cache invalidation ● No mature solutions yet ● Protocol-based (e.g. works only with HTTP)
- 74. ● Introduction ✔ ● Caching Architectural Patterns ○ Embedded ✔ ○ Embedded Distributed ✔ ○ Client-Server ✔ ○ Cloud ✔ ○ Sidecar ✔ ○ Reverse Proxy ✔ ○ Reverse Proxy Sidecar ✔ ● Summary Agenda
- 75. Summary
- 79. application-aware? Reverse Proxy Reverse Proxy Sidecar no yes no early adopter?
- 80. application-aware? Reverse Proxy Reverse Proxy Sidecar lot of data? security restrictions? yes no yes no early adopter?
- 81. application-aware? Reverse Proxy Reverse Proxy Sidecar lot of data? security restrictions? language-agnostic? yes no yes no no early adopter?
- 82. application-aware? Reverse Proxy Reverse Proxy Sidecar lot of data? security restrictions? language-agnostic? Embedded (Distributed) yes no yes no no no early adopter?
- 83. application-aware? Reverse Proxy Reverse Proxy Sidecar lot of data? security restrictions? language-agnostic? Embedded (Distributed) Sidecar yes no yes yes no no no early adopter?
- 84. application-aware? Reverse Proxy Reverse Proxy Sidecar lot of data? security restrictions? language-agnostic? Embedded (Distributed) Sidecar cloud? yes no yes yes yes no no no early adopter?
- 85. application-aware? Reverse Proxy Reverse Proxy Sidecar lot of data? security restrictions? language-agnostic? Embedded (Distributed) Sidecar cloud? Client-Server yes no yes yes yes no no no no early adopter?
- 86. application-aware? Reverse Proxy Reverse Proxy Sidecar lot of data? security restrictions? language-agnostic? Embedded (Distributed) Sidecar cloud? Client-Server Cloud yes no yes yes yes yes no no no no early adopter?
- 87. Resources ● Code for this talk: https://github.com/leszko/caching-patterns ● Hazelcast Sidecar Container Pattern: https://hazelcast.com/blog/hazelcast-sidecar-container-pattern/ ● Hazelcast Reverse Proxy Sidecar Caching Prototype: https://github.com/leszko/caching-injector ● NGINX HTTP Reverse Proxy Caching: https://www.nginx.com/resources/videos/best-practices-for-caching/