Scylla Summit 2018: The Short and Straight Road That Leads from Cassandra to Scylla
Download as PPTX, PDF1 like1,565 views
Felipe Móz, a big data specialist at Natura, discusses the transition from Cassandra to ScyllaDB to enhance performance and reduce costs. The migration significantly improved batch processing times and decreased operational expenses, showcasing ScyllaDB's advantages in flexibility and support. The document includes technical details on the hardware used and the outcomes of the transition.
Scylla Summit 2018: The Short and Straight Road That Leads from Cassandra to Scylla
- 1. The Short and Straight Road That Leads from Cassandra to Scylla Felipe Móz Big Data Specialist
- 2. Presenter bio First of all: I’m not a Jedi. I believe that without data you only have an opinion. I’m a big data specialist @Natura
- 3. Three brands, one vision
- 5. million Sales consultants 1.8 thousand 6.5 co-workers Some figures million consumers 100 8 countries Americas & Europe Natura stores 28 million Online consumers 3,5
- 6. TECH TALK
- 7. AWS Sensedia API Gateway OAM / AD Autenticação Scheduler Oracle Batch replication Private access + + Streaming + RDD Monitoring: Dynatrace ~40 long batch’s per day 12 x m4.4xlarge (autoscaling) ~10 builds/day +380k messages/day Streaming Statistics: ~160K completed rdd/day Processing Time: Avg: 76 ms Scheduling Delay Avg: 0 ms ARCHITECTURE
- 9. THE DECISION Why ScyllaDB? ▪ No more JVM, we are so glad; ▪ Lot of performance issues on C*/DSE; ▪ Support works! They know our use case and data modeling; ▪ Access directly to developers, if you need; ▪ They are flexible to contract: choose the right way to scale your db, by cores not nodes! ▪ Datastax overpricing during renewership process; ▪ ScyllaDB Roadmap is better than C*/DSE.
- 10. SCYLLA METRICS
- 11. C*/DSE METRICS
- 12. NSG RedeNaturaAzure-SP SUBNETNATPRD-AZ1-Isolated dc0prdvm0 (#0) Standard DS14 v2 (16 CPU, 112 GB DDR) LUN 11 x P20 SSD dc0prdvm1 (#1) Standard DS14 v2 (16 CPU, 112 GB DDR) LUN 11 x P20 SSD dc0prdvm3 (#3) Standard DS14 v2 (16 CPU, 112 GB DDR) LUN 11 x P20 SSD opscenterprd Standard DS1 v2 (1 CPU, 3,5 GB DDR) dc0prdvm4 (#4) Standard DS14 v2 (16 CPU, 112 GB DDR) LUN 11 x P20 SSD dc0prdvm5 (#5) Standard DS14 v2 (16 CPU, 112 GB DDR) LUN 11 x P20 SSD C*/DSE SIZING
- 13. dc0prdvm0 (#0) dc0prdvm1 (#1)dc0prdvm3 (#3) dc0prdvm4 (#4) dc0prdvm5 (#5) C*/DSE Cloud Hardware Costs US$ 2.008,44 Each node @Azure Location: Brazil Type: Ubuntu 16 LTS Size: DS14_V2 Managed Disks: 11 x P20 SSD Hours: 744h hours *Pricelist without reserve (DSE License not included) US$ 2.008,44 US$ 2.008,44 US$ 2.008,44 US$ 2.008,44
- 14. SCYLLADB SIZING NSG AWS SUBNETNATPRD-AZ1-Isolated node (#0) i3.4xlarge (16 CPU, 112 GB DDR) 3.5TB ephemeral NVME node (#1) i3.4xlarge (16 CPU, 112 GB DDR) 3.5TB ephemeral NVME node (#3) i3.4xlarge (16 CPU, 112 GB DDR) 3.5TB ephemeral NVME grafana t2.medium (1 CPU, 24GB) Node (#4) i3.4xlarge (16 CPU, 112 GB DDR) 3.5TB ephemeral NVME node (#5) i3.4xlarge (16 CPU, 112 GB DDR) 3.5TB ephemeral NVME
- 15. SCYLLADB Cloud Hardware Costs node (#0) node (#1)node (#3) Node (#4) node (#5) Each node @AWS Location: Brasil Type: ami-2c311640 (helpful) Size: i3.4xlarge Managed Disks: 2 x 1900 NVMe SSD Hours: 744h hours *Pricelist On-Demand (No Contract) (ScyllaDB License not included) US$ 913.54 US$ 913.54 US$ 913.54 US$ 913.54 US$ 913.54
- 16. RESULTS ▪ Reducing batch process at least to 10% ▪ In some cases the reductions went from 6 hours to below 10 minutes (1/36th!) ▪ Saving a lot of money ▪ Almost zero rebuilding ▪ From milliseconds to microseconds write latency Streaming Statistics: ~160K completed rdd/day Processing Time: Avg: 6 ms Scheduling Delay Avg: 0 ms Streaming Statistics: ~160K completed rdd/day Processing Time: Avg: 76 ms Scheduling Delay Avg: 0 ms DSE AVG 95th percentile ~ 220 msec by operations ScyllaDB AVG 95th percentile ~ 500 µsec by operations
- 17. TIPS ScyllaDB + Spark (MUST read) https://docs.scylladb.com/kb/scylla-and-spark-integration/ https://www.scylladb.com/2018/07/31/spark-scylla/ https://pt.slideshare.net/ScyllaDB/spark-powered-by-scylla
- 18. They aren't our provider, They are part of our team.
- 19. Thank You Any Questions ? Please stay in touch [email protected]