ShareChat’s Path to High-Performance NoSQL with ScyllaDB
0 likes5,325 views
🎥 Sign up for upcoming webinars or browse through our library of on-demand recordings here: https://www.scylladb.com/resources/webinars/ To power India’s leading social media platform, ShareChat needs to deliver impressive performance at rapidly-increasing scale – but without rapidly-increasing cost. With 180M monthly active users expecting real-time engagement with 2.5B posts per month, having the right database implementation is essential. Hear how Geetish Nayak, Staff Engineer/Architect - Platforms at ShareChat modernized the database powering their services to stay ahead of these challenges. Geetish will share the strategies they applied to improve performance 3-5x while reducing costs 50-80%. You will learn: - About their technical challenges related to supporting new locations, debugging, and performance at scale - What NoSQL capabilities were most critical for preparing their data architecture for the company’s next level of growth - How they measured the impact of their NoSQL migration, and what they have achieved so far - Best Practices that ShareChat followed when adopting ScyllaDB
Ad
More Related Content
Similar to ShareChat’s Path to High-Performance NoSQL with ScyllaDB (20)
More from ScyllaDB (20)
Ad
Recently uploaded (20)
Ad
ShareChat’s Path to High-Performance NoSQL with ScyllaDB
- 1. ShareChat’s Path to High-Performance NoSQL with ScyllaDB Geetish Nayak - Staff Engineer/Architect, Platforms
- 2. Poll Where are you in your NoSQL adoption?
- 3. ShareChat’s Path to High-Performance NoSQL with ScyllaDB Geetish Nayak - Staff Engineer/Architect, Platforms
- 4. + Infoworld 2020 Technology of the Year! + Founded by designers of KVM Hypervisor The Database Built for Gamechangers 4 “ScyllaDB stands apart...It’s the rare product that exceeds my expectations.” – Martin Heller, InfoWorld contributing editor and reviewer “For 99.9% of applications, ScyllaDB delivers all the power a customer will ever need, on workloads that other databases can’t touch – and at a fraction of the cost of an in-memory solution.” – Adrian Bridgewater, Forbes senior contributor + Resolves challenges of legacy NoSQL databases + >5x higher throughput + >20x lower latency + >75% TCO savings + DBaaS/Cloud, Enterprise and Open Source solutions + Proven globally at scale
- 5. +400 Gamechangers Leverage ScyllaDB Seamless experiences across content + devices India’s leading social media platform Corporate fleet management Real-time analytics 2,000,000 SKU -commerce management Fast computation of flight pricing Product catalog with over a billion items IoT for industrial machines Synchronize browser properties for millions Threat intelligence service using JanusGraph Real time fraud detection across 6M transactions/day Uber scale, mission critical chat & messaging app Network security threat detection Power ~50M X1 DVRs with billions of reqs/day Precision healthcare via Edison AI Inventory hub for retail operations Property listings and updates Unified ML feature store across the business Cryptocurrency exchange app Geography-based recommendations Distributed storage for distributed ledger tech Global operations- Avon, Body Shop + more Predictable performance for on sale surges GPS-based exercise tracking
- 6. Presenter Geetish Nayak - Staff Engineer/Architect, Platforms + 10 years of experience + Previously in engineering roles at Intuit and LucidWorks and also as a co-founder of Graphiti XYZ Inc. + Master’s degree in Information Systems from Carnegie Mellon (Pittsburgh)
- 10. - Detailed Metrics leading to effective debugging - Better performance - Low Latency / Higher Throughput - Supports DR as a first class citizen - More CONTROL even though a managed solution - Compaction Strategy - Changing Consistency Levels etc - Replication Factor - How much data to cache ? - Cost effective Why is a new database required ?
- 11. - A whitepaper compared ScyllaDB to one of the existing databases that we were using — the numbers were too good - We setup our own ScyllaDB cluster using the Kubernetes Operator provided by ScyllaDB team - Within ShareChat - Tested one of the biggest production workload in ShareChat against this ScyllaDB cluster Why we chose ScyllaDB
- 12. - Chat application - Real-Time Notification Framework - Counters - like, share, comments, other 10+ such counters - Ads Data Management Platform - In-Memory Database use cases are using ScyllaDB - Data Science Feature Store ShareChat ScyllaDB Use Cases at Scale
- 13. Architecture of Apache Kafka Counters Pipeline
- 14. Architecture of Data Science Feature Store
- 15. Architecture of Real-Time Communication Framework
- 16. - We have on-boarded around 70-80 TB of data already. Plan to onboard additional 50 TB - Almost 35-40 services are using ScyllaDB at scale - Our biggest cluster size is around 20 TB and our max throughput for one of our cluster is 1.2 Million Ops/sec ShareChat’s Usage of ScyllaDB
- 17. Counters Cluster ShareChat’s Usage of ScyllaDB
- 18. Notifications Cluster ShareChat’s Usage of ScyllaDB
- 19. - Metrics - better debuggability : We now have shard level metrics - suggestion for best practices : Range Queries, Token Awareness, Prepared Statements - query patterns - Cost - Min 50% - 80% reduction in database costs - Performance - Latency and Throughput - some of the workloads are getting latency numbers in microseconds - we are able to get more than 1 Mill Ops/sec for some of the clusters - CPU utilization goes beyond >80% and we see almost no degradation - Tech Guidance How ScyllaDB has Helped ShareChat
- 20. Metrics
- 21. Stability
- 22. - No sharing of ScyllaDB clusters between services - ScyllaDB Onboarding Document where Queries and Data Model gets reviewed by a team of experts at ShareChat - A single database driver(single interface) to talk to all of our databases; ScyllaDB being one of them - best practices such as use of prepared-statements, token-awareness etc are incorporated in the driver Some of the Best Practices ShareChat has Adopted
- 23. How we Planned to Migrate Data to ScyllaDB w/o Downtime
- 24. How we Planned to Migrate Data to ScyllaDB w/o Downtime
- 25. How we Planned to Migrate Data to ScyllaDB w/o Downtime
- 26. How we Planned to Migrate Data to ScyllaDB w/o Downtime
- 27. - ShareChat and ScyllaDB engineers almost work like a single team - Our database practices have certainly improved - We give a lot of thought to how we model our databases based on query patterns - Regular guidance from engineers has helped understand the pros and cons of ScyllaDB features - ScyllaDB Team takes away a lot of operational overhead from us - Detailed responses to tickets with root cause and mitigation steps has helped us Our Experience Working with Managed ScyllaDB
- 28. DevOps Team - Abhiroop Soni, Chandra Mohan Reddy, Jenson C S Platform Team - Chinmoy Mohapatra, Farhan Mohammad, Prateek Bhargav, Anuraj Jain Early Adopters - Ivan Burmitstrov, Karan Saluja , Mohit Dubey , Deepak Poptani, Himanshu Pandey, Charan Movva, Swapnil Bari, Shubham Sharma, Ritu Dhariwal Executives - Bhanu Singh, Manohar Charan, Gaurav Bhatia, Harshal Vora, Abraham Kandathil, Ankit Shah, Shailesh Kocchar, Chhaya Sharma, Abhishek Aggarwal Partners in Crime
- 29. Thank you for joining us today. @scylladb scylladb/ slack.scylladb.com @scylladb company/scylladb/ scylladb/
- 30. Q & A
- 31. Poll How much data do you under management of your transactional database?