Running a Cost-Effective DynamoDB-Compatible Database on Managed Kubernetes Services
0 likes204 views
Moreno Garcia, a solutions architect at Scylla, presents on deploying a cost-effective, DynamoDB-compatible database using Kubernetes, highlighting Scylla's performance advantages over Cassandra and DynamoDB. The document discusses Scylla's features, such as its compatibility with existing APIs, low latency, and self-optimizing capabilities, while introducing the Alternator API for flexible deployment options. The presentation covers deployment demos, associated challenges, and how to leverage Kubernetes for effective database management.
Running a Cost-Effective DynamoDB-Compatible Database on Managed Kubernetes Services
- 2. Running a Cost-effective DynamoDB Compatible Database on Managed Kubernetes Services
- 3. Presenter 3 Moreno Garcia Solutions Architect, Scylla Moreno Garcia is a problem solver by talent and trade. He has worked in IT for 17 years, the last seven as a solution architect. An open source software enthusiast, Moreno loves applying new technology to solve old problems, and hates acronyms with a passion.
- 4. Agenda 4 + Why Scylla? + Why and What is Alternator? + Why and What is Scylla Operator? + Demo with an interactive game of Tic Tac Toe + Q&A
- 5. 5 + The Real-Time Big Data Database + Drop-in replacement for Apache Cassandra and Amazon DynamoDB + 10X the performance & low tail latency + Open source, enterprise and cloud options + Founded by the creators of KVM hypervisor + HQs: Palo Alto, CA, USA; Herzelia, Israel; Warsaw, Poland About ScyllaDB
- 6. Lower Node Count Millions of OPS per node of throughput reduces the number of nodes your application requires. Why Scylla? Predictable, Low Latencies Consistent single-digit millisecond p99 latencies. Less Complexity Smaller footprint, self-optimizing, works out-of-the-box 6 Cassandra/DynamoDB Compatibility Drop-in replacement, requires no changes to client code
- 7. Scylla Design Advantages + Efficient implementation for modern hardware + Throughput 10x higher than Cassandra + Linear scalability to many-core machines + Focused on modern fast SSDs + Compatible with Cassandra and its APIs (CQL, Thrift) + All things asynchronous + Built-in schedulers + Self-optimizing database (minimal configuration)
- 8. ▪ Simplified deployments • Eliminated cache ▪ Reduced infrastructure • 962 C* nodes → 78 Scylla nodes ▪ Performance improvements • 95% better p99 latencies 8
- 9. A Match Made in Heaven + Great database + Requires operational expertise + Great workload management platform Can we leverage Kubernetes to write a great management layer for Scylla ? 9
- 11. Scylla’s DynamoDB-compatible API + Scale-out system with complete observability + Deployment flexibility + On prem, multi-cloud, hybrid + Open source or Scylla Cloud + No vendor lock-in + Better performance and less expensive
- 12. Much Better Price Performance Managed Scylla (“Scylla Cloud”) Is 5 times less expensive than DynamoDB. See the benchmark https://www.scylladb.com/product/benchmarks/dynamodb-benchmark/
- 13. Alternator Deployment Options Install in Your Datacenter ➔ Scylla Open Source ➔ Scylla Enterprise Install at a Cloud Provider ➔ Scylla Open Source ➔ Scylla Enterprise Database as a Service ➔ Fully managed Scylla clusters ➔ 24*7 maintenance and support. On-Prem Cloud Hosted Scylla Cloud 13
- 14. Alternator Support Level* Fully Supported Mostly Supported No (yet) Supported Table Operations ✓ Item Operations ✓ Batch Operations ✓ Scans ✓ Tags ✓ Secondary Indexes ✓ TTL ✓ Streams ✓ Transactions ✓ * Alternator: DynamoDB API in Scylla | Scylla Docs
- 15. Alternator Support Level* Supported Authentication ✓ Backup / Restore NC Tracing NC Monitoring NC Alerts NC * Alternator: DynamoDB API in Scylla | Scylla Docs
- 16. Alternator is GA, Try it now! docker run --name some-scylla -d -p 8000:8000 scylladb/scylla:4.1.0 --alternator-port=8000 --alternator-write-isolation=always aws --endpoint-url 'http://127.0.0.1:8000' dynamodb create-table --table-name MusicCollection --attribute-definitions AttributeName=Artist,AttributeType=S AttributeName=SongTitle,AttributeType=S --key-schema AttributeName=Artist,KeyType=HASH AttributeName=SongTitle,KeyType=RANGE --provisioned-throughput ReadCapacityUnits=5,WriteCapacityUnits=5
- 18. Scylla on K8s
- 19. StatefulSet Rack N, Datacenter M ... Cluster (CRD) Custom Resource Definition Member Services (Static IP) Controller communication through Labels / Annotations Member Services (Static IP) Member Services (Static IP) writewatch StatefulSet Rack 1, Datacenter 1 StatefulSet Rack 1, Datacenter 2 19
- 20. Sidecar - Controller CRD + Controller + Sidecar Sidecar, Manager Agent Pod Sidecar needed to: + Setup config files + Install plugins at startup + Backup and Restore functionality + Future extensibility Member 20
- 21. Operator = Controller(s) + CRD(s) Controller Observe Calculate Reconcile write Our Goal
- 22. Mapping of Abstractions Member/Node Pod Cluster Rack Datacenter StatefulSet StatefulSets Cluster Custom Resource (CRD) 22
- 23. Demo Interactive Tic Tac Toe game 23
- 24. Deployment Deploy Scylla Operator Deploy Scylla cluster Deploy Scylla Monitoring Stack Stress the Deployment Common Procedures Upgrade from 4.0.0 -> 4.0.1 Scale Down24 + Deployment - GKE + Deploy Scylla Operator + Deploy Scylla cluster + Deploy Scylla Monitoring Stack + Stress the Deployment (Tic Tac Toe!) + Common Procedures + Scale Down + Upgrade from 4.0.0 -> 4.0.1 + Demo Code Source: AWS docs Demo
- 25. Status 25
- 26. Scylla Alternator + Production ready from Scylla Open Source 4.0 + Docker, AMI, Scylla Cloud + Coming soon to Scylla Enterprise 2020.1.0 + Scylla Alternator | Scylla Docs 26
- 27. Kubernetes Operator Easy to get started: https://github.com/scylladb/scylla-operator 27
- 28. Scylla Operator - Supported + Topology Changes + Proper token ring initialization, with only one node bootstrapping at a time + Down scale + Out Scale + Cluster Management + Managed Configuration, including rolling update + Rolling Upgrades / Downgrades + Integrations + Prometheus / Grafana monitoring Stack using Helm Charts + Secured 28
- 29. Scylla on K8s - In the works! + Multiple Data Centers Support + Auto Scaling + CI/CD + Support CSR + Scylla manager integration (repairs, backups) + Performance benchmarks on EKS, GKS + Scylla as K8s Service + Helm packaging + Your Feature Here! 29
- 30. Resources 30
- 32. United States 2445 Faber St, Suite #200 Palo Alto, CA USA 94303 Israel 11 Galgalei Haplada Herzelia, Israel www.scylladb.com @scylladb Thank you