Moving from DynamoDB to ScyllaDB: Technical Comparison and the Path to Success
0 likes209 views
The document presents a technical comparison between DynamoDB and ScyllaDB, highlighting ScyllaDB's advantages in performance, cost, and scalability. It discusses migration pathways, case studies, and best practices for transitioning from DynamoDB to ScyllaDB, emphasizing minimal changes to applications. The conclusion suggests that migrating to ScyllaDB can provide significant benefits and encourages exploring available resources for successful implementation.
Moving from DynamoDB to ScyllaDB: Technical Comparison and the Path to Success
- 1. DynamoDB to ScyllaDB A technical comparison and path to success Tim Koopmans, Product Experience Guilherme Nogueira, Technical Director
- 2. Presentation Agenda + Introduction + Understanding ScyllaDB + Comparing DynamoDB and ScyllaDB + Migration Options and Tools + Case Studies and Best Practices + Conclusion
- 4. Why ScyllaDB? On-Prem Cloud Hosted Scylla Cloud Best High Availability in the industry Best Disaster Recovery in the industry Best scalability in the industry Best Performance in the industry Auto-tune - out of the box performance Fully compatible with Cassandra & DynamoDB Best Elasticity in the industry Best Price/Performance in the industry
- 5. Tablets Schedulers Shard per Core Tiered Storage Built in caching, Load balancing What sets us apart?
- 6. Why do DynamoDB customers switch?
- 7. + Adequate provisioned capacity but high throttling + Throttling despite activated AWS Application Auto Scaling + Throttling in on-demand capacity mode + Presence of a hot partition + Traffic exceeding account throughput quotas ProvisionedThroughputExceededException
- 9. The benchmark Provisioned Throughput 200K RCU 250K WCU Cluster Size i4i.2XL x 3 YCSB Loaders c5.2XL x 8 18-150K ops/sec lp.scylladb.com/price-performance-dynamoDB-benchmark-offer
- 10. Latency
- 11. Latency
- 12. Throughput
- 13. Throughput + DynamoDB + Throughput limited at partition level + ScyllaDB + Balanced / prioritized workloads across the entire cluster
- 14. Cost DynamoDB ScyllaDB Provisioned cost $30k/mo $4.5k/mo On-demand cost $206k/mo* $4.5k/mo P99 latency 20.2 ms 11.1 ms
- 16. + ScyllaDB is closely related to DynamoDB Similarities Dynamo Paper (Amazon) BigTable Paper (Google) 2006 2007 2008 2015 2012
- 17. + ScyllaDB implements CQL + Query language + Protocol + Data model + Alternator brought in DynamoDB API compatibility + Not a translation layer! + Popular standards as transport + HTTP/S and JSON What ScyllaDB brings to the table
- 18. Comparison Aspect CQL DynamoDB API Transport CQL binary protocol HTTP/S and JSON Protocol Cluster-aware Single endpoint Query language Prepared statements No prepared statements Write model Conflict-free Replicated Data Types (CRDT) / Lightweight Transactions (LWT) Read-Modify-Write
- 19. Comparison - Schema + CQL enforces a schema + Partition Key(s), Clustering Key(s) + CQL types + UDTs + Materialized Views, Local Secondary Indexes, Global Secondary Indexes + CDC + DynamoDB enforces Keys + Primary Key, Sorting Key + DynamoDB Types + Local Secondary Indexes, Global Secondary Indexes + DynamoDB Streams
- 20. Depending on the chosen path + DynamoDB to ScyllaDB Alternator + No application rewrite + Table should work as-is + Run DDB-compatible workloads anywhere! + DynamoDB to CQL + Application and Table changes required + Feature-rich, cluster aware + High performance Migration Paths
- 21. + DynamoDB to ScyllaDB Alternator + Use DynamoDB-API compatible libraries and tools + Dual writes + Change application to target both DDB and Alternator + Leverage DynamoDB Streams + Lambda + Historical data lift-and-shift + Read from DDB table + Consumes RCUs + Beware of throttling and $ + Export to S3 Migration Paths - DDB to Alternator
- 22. + DynamoDB to ScyllaDB using CQL + Use DynamoDB-API compatible tools + Python boto3 and Java AWS SDK + Native CQL libraries for interacting with ScyllaDB + Dual writes + Change application to target both DDB and ScyllaDB + Historical data lift-and-shift + ETL from DDB types to CQL types + Read from table or S3 Migration Paths - DynamoDB to CQL
- 23. Libraries designed to work with DynamoDB API works with Alternator + Python boto3, Java AWS SDK… + Kafka Streams Connector + NoSQL Workbench + Spark Tooling - Alternator
- 24. Tooling - CQL + Built-in shard awareness on ScyllaDB drivers + Enabled by default + Supports many languages: + Leveraged by connectors: + Rack awareness (save on Cloud networking)
- 25. + Reading tables can trip throttling + Impact to Production apps + Reading JSON from S3 has its challenges + Not everything abides to JSON standards + Better to stick with official libraries to import data + Schema changes over the years Challenges
- 26. Case Studies And Best Practices
- 27. + Cost + Limited SDK + Write throughput + Why ScyllaDB? + Multi-region + ICS, CDC, Seastar shared-nothing, drivers + 1ms writes, single-digit read p99 Why did Disney Streaming move out of DynamoDB?
- 28. Real World Examples Digital Turbine migrated from DynamoDB to ScyllaDB Alternator on GCP + Ease of migration + No application refactor + Reduce cost (20% cost reduction upfront, with opportunity for more) + Better performance + ScyllaDB Alternator cluster, Cluster ~2,000 ops/sec (90% reads , 10% Writes) + Leveraging DynamoDB API + DynamoDB ~1,400 ops/sec (90% reads , 10% Writes) + Observed light throttling on the DynamoDB side
- 29. Real World Examples Yieldmo migrated from DynamoDB to ScyllaDB Alternator on AWS + Billions of ad auctions per day – in milliseconds + No room for error + No code changes required + Improve latencies + Reduce cost
- 30. + Keep vital metrics close + Use Scylla Monitoring stack + Know your data and your application + Learn about historical changes to your application + Keep it simple Best Practices
- 31. + Know the technology + Use the best tool for the job + Apples to Apples migration might be easier + Migrating to a different protocol might open many possibilities + PoC the technology + Application integration, smoke tests + Performance comparison + Cost Do’s and Don’ts
- 32. + Use the best tool for the job + When interacting with DDB use default libraries + CQL Shard-aware drivers + Full compatibility is impossible + Direct S3 export + Lambda triggers + CloudFormation integration for creating tables Do’s and Don’ts - Cont'd
- 33. Conclusion
- 34. Conclusion + ScyllaDB University is 100% FREE + Lessons on Alternator and CQL + Migration + Many useful blogs + Customer success stories + Migrating from DynamoDB to ScyllaDB can be extremely beneficial + Easy to get started + Also available as a Managed Service
- 35. A ScyllaDB Community Extreme Scale Engineering | ONLINE | MARCH 11 + 12
- 36. Thank you for joining us today. @scylladb scylladb/ slack.scylladb.com @scylladb company/scylladb/ scylladb/