RedisConf18 - Ultra Scaling with Redis Enterprise

May 30, 20181 like213 views

This document discusses various ways to scale Redis as needs increase. It outlines common bottlenecks like max memory or CPU usage on individual shards. Solutions proposed include re-sharding to use multiple cores, scaling nodes up or out by replacing or adding nodes, using Redis on Flash to overcome memory limitations, and employing multi-proxy or cluster APIs to overcome network saturation. The overall message is that Redis Enterprise allows scaling Redis seamlessly and effortlessly through different techniques without requiring code changes.

The Need to Scale is a Good Thing!
It means you are doing something right…

You just need to figure out HOW to scale…
…Fast… Seamlessly… Effortlessly…

Problem: Redis Shard Hits Max Memory
• Increase memory?
• Merely buys time until the next round of expansion
• Eventually you’ll hit the node maximum capacity
• Scale the machine up?
• Expensive and inefficient

Problem: Shard Hits Max CPU
• Are you using Redis efficiently?
• Inefficient data types
• Heavy commands (KEYS, SMEMBERS, HGETALL, LRANGE,
etc.)
• Is your architecture designed for scale?

Solution: Re-Sharding!
• Re-Sharding Redis Labs
• Instantly improve performance
• Running the same load on multiple cores
• No code change required

Problem: Node’s Resources Are Saturated
• CPU
• Memory
• Network

Solution: Scale Node – Up or Out
• Seamless scale up
• Replace the node to a stronger node
• Seamless scale out
• Add nodes to the cluster and re-balance

Solution: Redis on Flash (ROF)
• Using NVMe SSD as an extension of RAM
• Overcome RAM size limitation
• Reduce the hardware costs
• No code change required

Solution: Overcome Network Saturation
• Multi-Proxy
• Allows transparent scaling with no application changes
• OSS Cluster API
• Allows linear scalability, using "smart client"

Let’s Recap - Scaling with Redis Enterprise
• Implementation efficiency - data structures and commands
• Architecture – logical separation
• Seamless scale up and out
• Redis on Flash
• Multi proxy
• OSS Cluster API

RedisConf18 - Ultra Scaling with Redis Enterprise

Redis CRDTs allow for active-active geo-distributed applications by providing smart and transparent conflict resolution for concurrent writes across data centers. CRDTs use conflict-free replicated data types and causal consistency to ensure eventual consistency while maintaining high availability. Common data types like counters, sets, strings and lists implement developer-defined conflict resolution, such as last write wins, sum of values, and preserving all items. This allows active-active applications to be built simply using standard Redis commands and datatypes without custom conflict handling code.

RedisConf18 - Techniques for Synchronizing In-Memory Caches with RedisRedis Labs

The document describes techniques for synchronizing in-memory caches across multiple web servers using Redis. It discusses the problems with traditional in-process caches, such as data lag and inconsistency. The proposed solution uses Redis for the source of truth, with each web server maintaining an in-process cache. Hash slots are used to partition keys and publish updates via Redis pub/sub. When a key is requested, the server checks if its cache version is stale by comparing timestamps. This approach maintains consistency while minimizing network usage.

RedisConf18 - Serving Automated Home Valuation with Redis & KafkaRedis Labs

This document summarizes the design iterations of an automated home valuation system at Redfin that uses Redis and Kafka. It describes moving the system from AWS to using Kafka and Samza for distributed streaming. Key points included prefetching estimate values for nearby properties to warm the cache, increasing the cache hit rate from 55% to 80%. Monitoring and decoupling systems were identified as areas for further improvement. The system helped reduce response times and API calls for home valuations.

RedisConf18 - Re-architecting Redis-on-Flash with Intel 3DX Point™ MemoryRedis Labs

The document discusses re-architecting Redis-on-Flash with Intel 3D XPoint memory. It introduces 3D XPoint memory as a new type of memory that is persistent, has high capacity of 6 TB per system, and is cheaper than DRAM. RedisLabs and Intel are collaborating to build the next version of Redis-on-Flash using 3D XPoint memory to increase scalability through larger memory modules and reduce costs compared to DRAM. The challenges include higher latency compared to DRAM and evolving standards.

RedisConf18 - Redis at LINE - 25 Billion Messages Per DayRedis Labs

LINE uses Redis for caching and primary storage of messaging data. It operates over 60 Redis clusters with over 1,000 machines and 10,000 nodes to handle 25 billion messages per day. LINE developed its own Redis client and monitoring system to support client-side sharding without a proxy, automated failure detection, and scalable cluster monitoring. While the official Redis Cluster was tested, it exhibited some issues around memory usage and maximum node size for LINE's large scale needs.

RedisConf18 - Microservicesand Redis: A Match made in HeavenRedis Labs

Redis is being used in multiple ways to support the microservices architecture at eharmony. Redis is used as an authentication store for access and refresh tokens. It is also used as the primary data store for user profiles, activities, and badges. Redis hashes and sorted sets enable fast retrieval and updates of user data and security anomalies. Redis pub/sub is utilized for dynamic configuration updates. Sentinel provides high availability. Redis is helping to improve performance and scalability of the microservices.

RedisConf18 - Redis as a time-series DBRedis Labs

Running Analytics at the Speed of Your BusinessRedis Labs

The speed at which you can extract insights from your data is increasingly a competitive edge for your business. Data and analytics have to be at lightning fast speeds to seriously impact your user acquisition. Join this webinar featuring Forrester analyst Noel Yuhanna and Leena Joshi, VP Product Marketing at Redis Labs to learn how you can glean insights faster with new open source data processing frameworks like Spark and Redis. In this webinar you will learn: * Why analytics has to run at the real time speed of business * How this can be achieved with next generation Big Data tools * How data structures can optimize your hybrid transaction-analytics processing scenarios

RedisConf18 - Redis on FlashRedis Labs

The document discusses Etermax's use of Redis as their primary database. They initially used MySQL but faced latency issues as user base and traffic grew. They migrated to Redis, starting with a single instance and eventually using Redis Enterprise Cloud with sharding. Testing showed Redis on Flash provided acceptable latency while significantly reducing infrastructure costs. Migrating production to Redis on Flash reduced costs by over 50% without code changes. Redis on Flash works well if instant data recovery is not required.

RedisConf18 - Open Source Built for Scale: Redis in Amazon ElastiCache ServiceRedis Labs

This document summarizes Andi Gutmans' presentation about Redis and Amazon ElastiCache. The key points are: 1. Amazon ElastiCache provides a fully managed Redis compatible data store for building low latency and highly scalable applications on AWS. 2. AWS contributes to open source Redis projects through bug fixes, feature additions and code improvements. Recent contributions include improvements to replication, migration and scaling. 3. A new open source project was announced to add encryption-in-transit to Redis, allowing easy encryption of client-server and cluster communications without requiring application changes. 4. AWS is contributing encryption functionality to the core Redis codebase through an open pull request.

RedisConf18 - Redis Enterprise on Cloud Native Platforms Redis Labs

This document provides an introduction to cloud-native platforms and Kubernetes, and demonstrates how Redis Enterprise can run on these platforms. It discusses how Kubernetes provides orchestration of containers and manages the application lifecycle. It then demonstrates deploying Redis Enterprise on Kubernetes, showing how it uses a custom Kubernetes controller and operator to provide auto-bootstrapping of Redis clusters within Kubernetes pods. The demo shows creating a Redis database, service discovery, and benchmarking tool deployment on the Kubernetes-hosted Redis Enterprise clusters.

RedisConf18 - Writing modular & encapsulated Redis codeRedis Labs

This document discusses writing modular and encapsulated Redis code. It introduces the concepts of promises and futures as they relate to Redis pipelines and transactions. Promises schedule Redis operations, returning future objects that can be transformed and bound to variables. This approach separates data access logic from business logic, making the code more readable, maintainable and refactorable compared to do-everything functions.

RedisConf18 - Redis Fault InjectionRedis Labs

This document discusses Redis Fault Injection (RedFI), a tool for introducing faults during testing to simulate error conditions that may not otherwise occur. It provides fault injection as a proxy to Redis in a transparent way. Example use cases include testing race conditions, latency, dropped connections, and slow consumers. The document demonstrates RedFI and outlines future plans to integrate similar fault injection into Envoy and build replay and regression detection systems.

Kafka at scale facebook israelGwen (Chen) Shapira

This document provides guidance on scaling Apache Kafka clusters and tuning performance. It discusses expanding Kafka clusters horizontally across inexpensive servers for increased throughput and CPU utilization. Key aspects that impact performance like disk layout, OS tuning, Java settings, broker and topic monitoring, client tuning, and anticipating problems are covered. Application performance can be improved through configuration of batch size, compression, and request handling, while consumer performance relies on partitioning, fetch settings, and avoiding perpetual rebalances.

RedisConf18 - Common Redis Use Cases for Cloud Native Apps and MicroservicesRedis Labs

The document discusses cloud native applications and architectures. It explains that cloud native applications are designed to take advantage of cloud computing abilities like scaling up and down on demand and limitless compute. Being cloud native isn't just an architectural pattern but a combination of practices like agile development, continuous delivery, automation, containers, and microservices that change development culture and move to a DevOps model. It also discusses concepts like serverless computing, functions as a service, microservices, and twelve factor applications.

Walmart & IBM Revisit the Linear Road Benchmark- Roger Rea, IBMRedis Labs

The document compares the performance of IBM Streams to other streaming analytics offerings using the Linear Road benchmark, finding that IBM Streams achieved an L-Rating of 200 using 4 Azure nodes, significantly outperforming Apache Apex and Apache Storm. It also describes how Walmart uses streaming analytics for real-time inventory control and logistics monitoring, and how IBM Streams was able to implement the Linear Road benchmark in under 15 days of development time.

What's new with enterprise Redis - Leena Joshi, Redis LabsRedis Labs

Redis Labs manages over 160k+ HA databases, 10k clustered databases, without data loss in spite of one node failure a day and one data center outage per month. Using Enterprise Redis(RLEC), Redis Labs delivers seamless zero downtime scaling, true high availability with persistence, cross-rack/zone/ datacenter replication and instant automatic failover. Learn how. Join this session for a deep dive into how enterprise Redis makes for no-hassle Redis deployments and the roadmap for new Redis capabilities. Discover new cost savings with Redis on Flash for cost-effective high performance operations and analytics

Scaling Redis Cluster Deployments for Genome Analysis (featuring LSU) - Terry...Redis Labs

RedisConf18 - My Other Car is a Redis ClusterRedis Labs

This document discusses using Redis as both a cache and primary data store. Redis is described as fast, simple, and easily scalable. It can be used as a cache for things like user profiles, with hot data stored in Redis and cold data stored elsewhere, like LMDB. Redis is also used as a primary store for tracking metrics like pageviews and purchases. The document provides examples of storing hyperloglog data in Redis to track unique counts and expiries. It also discusses techniques for load balancing and aggregating Redis data.

The Last Pickle: Distributed Tracing from Application to DatabaseDataStax Academy

Monitoring provides information on system performance, however tracing is necessary to understand individual request performance. Detailed query tracing has been provided by Cassandra since version 1.2 and is invaluable when diagnosing problems. Although knowing what queries to trace and why the application makes them still requires deep technical knowledge. By merging Application tracing via Zipkin and Cassandra query tracing we automate the process and make it easier to identify and resolve problems. In this talk Mick Semb Wever, Team Member at The Last Pickle, will introduce Cassandra query tracing and Zipkin. He will then propose an extension that allows clients to pass a trace identifier through to Cassandra, and a way to integrate Zipkin tracing into Cassandra. Driving all this is the desire to create one tracing view across the entire system.

PlayStation and Searchable Cassandra Without Solr (Dustin Pham & Alexander Fi...DataStax

When we were preparing for PlayStation4 launch we faced 1 hard problem: how to make our games and videos storage system super-fast, highly available and fault tolerant. Moving from relation database to Cassandra is not easy, and it even harder if you want to support different search and query use cases. Join our talk to learn how we managed to build highly available platform, that supports tens of millions of active users and can execute multiple user specific queries in less than a millisecond. And all of it without using Solr or ElasticSearch. About the Speaker Alexander Filipchik, Sony Alex spent last 4 years of his life building the next generation of PlayStation Network. He is honored to be a part of a small team of engineers who managed to build from scratch a platform that scaled from 0 users to 1 million PS4s in just 1 day and have being landing 1.5 million of new devices per month since and now reached tenth of millions of active users. He is passionate about technology, innovations, walking his dog and building scalable software using Cassandra.

Strata+Hadoop 2017 San Jose: Lessons from a year of supporting Apache Kafkaconfluent

The number of deployments of Apache Kafka at enterprise scale has greatly increased in the years since Kafka’s original development in 2010. Along with this rapid growth has come a wide variety of use cases and deployment strategies that transcend what Kafka’s creators imagined when they originally developed the technology. As the scope and reach of streaming data platforms based on Apache Kafka has grown, the need to understand monitoring and troubleshooting strategies has as well. Dustin Cote and Ryan Pridgeon share their experience supporting Apache Kafka at enterprise-scale and explore monitoring and troubleshooting techniques to help you avoid pitfalls when scaling large-scale Kafka deployments. Topics include: - Effective use of JMX for Kafka - Tools for preventing small problems from becoming big ones - Efficient architectures proven in the wild - Finding and storing the right information when it all goes wrong Visit www.confluent.io for more information.

Webinar: Diagnosing Apache Cassandra Problems in ProductionDataStax Academy

This document provides guidance on diagnosing problems in Cassandra production systems. It recommends first using OpsCenter to identify issues, then monitoring servers, applications, and logs. Common problems discussed include incorrect timestamps, tombstones slowing queries, not using a snitch, version mismatches, and disk space not being reclaimed. Diagnostic tools like htop, iostat, and nodetool are presented. The document also covers JVM garbage collection profiling to identify issues like early object promotion and long minor GCs slowing the system.

Azure + DataStax Enterprise Powers Office 365 Per User StoreDataStax Academy

Day 2 General Session Presentations RedisConfRedis Labs

The document discusses new memory technologies like persistent memory and their implications. It provides latency and bandwidth numbers for different memory types and notes that heterogeneous memory systems using tiers of DRAM and NVM provide opportunities for better performance and cost. Examples are given of key-value stores and databases leveraging NVM to achieve high performance while reducing costs. The talk also discusses how new distributed data structures like CRDTs could be used across servers with shared memory.

Tsinghua University: Two Exemplary Applications in ChinaDataStax Academy

In this talk, we will share the experiences of applying Cassandra with two real customers in China. In the first use case, we deployed Cassandra at Sany Group, a leading company of Machinery manufacturing, to manage the sensor data generated by construction machinery. By designing a specific schema and optimizing the write process, we successfully managed over 1.5 billion historical data records and achieved the online write throughput of 10k write operations per second with 5 servers. MapReduce is also used on Cassandra for valued-added services, e.g. operations management, machine failure prediction, and abnormal behavior mining. In the second use case, Cassandra is deployed in the China Meteorological Administration to manage the Meteorological data. We design a hybrid schema to support both slice query and time window based query efficiently. Also, we explored the optimized compaction and deletion strategy for meteorological data in this case.

HIgh Performance Redis- Tague Griffith, GoProRedis Labs

High Performance Redis looks at a wide range of techniques - from programming to system tuning - to deploy and maintain an extremely high performing Redis cluster. From the operational perspective, the talk lays out multiple techniques for clustering (sharding) Redis systems and examines how the different approaches impact performance time. The talk further examines system settings (Linux network parameters, Redis system) and how they impact performance (both good and bad). Finally, for the developer, we look at how different ways of structuring data actually demonstrate different performance characteristics

DataEngConf SF16 - BYOMQ: Why We [re]Built IronMQHakka Labs

The document discusses reasons for reimplementing messaging queue functionality instead of using an existing solution like RabbitMQ or Kafka. It notes that reimplementing provides full understanding and control, allowing for quick fixes and easy addition of features without needing a daemon. However, it also acknowledges that reimplementing takes a long time to reach stability. The document then describes the specific requirements for the reimplemented messaging queue and provides a brief overview of its design and performance.

Scaling a High Traffic Web Application: Our Journey from Java to PHP120bi

What makes an application scale? What should you worry about early on and what can wait? Over the last 3 years, Achievers has learned many lessons and gained fundamental knowledge on scaling our SaaS platform. CTO Dr. Aris Zakinthinos will present and discuss the decisions we’ve made including language choice, server architecture, and much more; join us while we share tips, tricks, and things to absolutely avoid. Throughout the evening you will have the opportunity to talk to the development team behind the Achievers Platform and ask questions on scaling best practices.

Scaling High Traffic Web ApplicationsAchievers Tech

More Related Content

What's hot (20)

RedisConf18 - Redis on FlashRedis Labs

RedisConf18 - Open Source Built for Scale: Redis in Amazon ElastiCache ServiceRedis Labs

RedisConf18 - Redis Enterprise on Cloud Native Platforms Redis Labs

RedisConf18 - Writing modular & encapsulated Redis codeRedis Labs

RedisConf18 - Redis Fault InjectionRedis Labs

Kafka at scale facebook israelGwen (Chen) Shapira

RedisConf18 - Common Redis Use Cases for Cloud Native Apps and MicroservicesRedis Labs

Walmart & IBM Revisit the Linear Road Benchmark- Roger Rea, IBMRedis Labs

What's new with enterprise Redis - Leena Joshi, Redis LabsRedis Labs

Scaling Redis Cluster Deployments for Genome Analysis (featuring LSU) - Terry...Redis Labs

RedisConf18 - My Other Car is a Redis ClusterRedis Labs

The Last Pickle: Distributed Tracing from Application to DatabaseDataStax Academy

PlayStation and Searchable Cassandra Without Solr (Dustin Pham & Alexander Fi...DataStax

Strata+Hadoop 2017 San Jose: Lessons from a year of supporting Apache Kafkaconfluent

Webinar: Diagnosing Apache Cassandra Problems in ProductionDataStax Academy

Azure + DataStax Enterprise Powers Office 365 Per User StoreDataStax Academy

Day 2 General Session Presentations RedisConfRedis Labs

Tsinghua University: Two Exemplary Applications in ChinaDataStax Academy

HIgh Performance Redis- Tague Griffith, GoProRedis Labs

DataEngConf SF16 - BYOMQ: Why We [re]Built IronMQHakka Labs

RedisConf18 - Redis on FlashRedis Labs

RedisConf18 - Open Source Built for Scale: Redis in Amazon ElastiCache ServiceRedis Labs

RedisConf18 - Redis Enterprise on Cloud Native Platforms Redis Labs

RedisConf18 - Writing modular & encapsulated Redis codeRedis Labs

RedisConf18 - Redis Fault InjectionRedis Labs

Kafka at scale facebook israelGwen (Chen) Shapira

RedisConf18 - Common Redis Use Cases for Cloud Native Apps and MicroservicesRedis Labs

Walmart & IBM Revisit the Linear Road Benchmark- Roger Rea, IBMRedis Labs

What's new with enterprise Redis - Leena Joshi, Redis LabsRedis Labs

Scaling Redis Cluster Deployments for Genome Analysis (featuring LSU) - Terry...Redis Labs

RedisConf18 - My Other Car is a Redis ClusterRedis Labs

The Last Pickle: Distributed Tracing from Application to DatabaseDataStax Academy

PlayStation and Searchable Cassandra Without Solr (Dustin Pham & Alexander Fi...DataStax

Strata+Hadoop 2017 San Jose: Lessons from a year of supporting Apache Kafkaconfluent

Webinar: Diagnosing Apache Cassandra Problems in ProductionDataStax Academy

Azure + DataStax Enterprise Powers Office 365 Per User StoreDataStax Academy

Day 2 General Session Presentations RedisConfRedis Labs

Tsinghua University: Two Exemplary Applications in ChinaDataStax Academy

HIgh Performance Redis- Tague Griffith, GoProRedis Labs

DataEngConf SF16 - BYOMQ: Why We [re]Built IronMQHakka Labs

Similar to RedisConf18 - Ultra Scaling with Redis Enterprise (20)

Scaling a High Traffic Web Application: Our Journey from Java to PHP120bi

Scaling High Traffic Web ApplicationsAchievers Tech

Архитектура приложений с использованием MySQL, Петр Зайцев (Percona)Ontico

The document discusses architecture patterns for MySQL powered applications. It covers common patterns used 10 years ago like load balancing, caching, and sharding. It also discusses how modern hardware, software, cloud computing, and automation have changed architectures. The document provides guidance on questions to consider for architectures like scale, high availability, and team experience. It also summarizes basic architecture patterns including single instance, high availability, and scaling reads, writes, and data size.

Redis - The Universal NoSQL ToolEberhard Wolff

Redis is a fast, in-memory key-value store that can be used as a cache, message broker, and centralized locking system. It stores data in memory for high performance, but can also asynchronously write data to disk for persistence. Redis supports data types like strings, lists, sets and sorted sets. It enables high availability through master-slave replication and horizontal scaling through sharding of data across multiple nodes.

Webinar: Migrating from RDBMS to MongoDBMongoDB

This document discusses migrating from an RDBMS to MongoDB. It begins by introducing the presenter and stating the goal is to explore issues in moving an existing RDBMS system to MongoDB. It then discusses determining the value of migrating, roles and responsibilities, bulk migration techniques, and approaches to cutting over the system. Key points made include understanding why you want to migrate, assessing the effort required versus the potential pain relief, involving all roles including developers and DBAs, using tools like mongoimport for bulk loads, and testing before any production cutover.

Make Life Suck Less (Building Scalable Systems)Bradford Stephens

Redis Everywhere - Sunshine PHPRicard Clau

Redis is an in-memory key-value data store that can be used for caching, sessions, queues, leaderboards, and more. It provides fast performance due to being memory-resident and supporting different data structures like strings, hashes, lists, sets, and sorted sets. Redis is useful for read-heavy and real-time applications but may not be suitable if data does not fit in memory or for relational data needs. The presentation discusses using Redis with PHP and Symfony, data sharding strategies, and war stories from a social game with 7.5M daily users.

Make Life Suck Less (Building Scalable Systems)guest0f8e278

Rapid Cluster Computing with Apache Spark 2016Zohar Elkayam

Big Data in a Public CloudCloudSigma

Things Every Oracle DBA Needs to Know about the Hadoop EcosystemZohar Elkayam

Session from BGOUG I presented in June, 2016 Big data is one of the biggest buzzword in today's market. Terms like Hadoop, HDFS, YARN, Sqoop, and non-structured data has been scaring DBA's since 2010 - but where does the DBA team really fit in? In this session, we will discuss everything database administrators and database developers needs to know about big data. We will demystify the Hadoop ecosystem and explore the different components. We will learn how HDFS and MapReduce are changing the data world, and where traditional databases fits into the grand scheme of things. We will also talk about why DBAs are the perfect candidates to transition into Big Data and Hadoop professionals and experts.

NoSQLDatabasesAdi Challa

The document discusses the history and concepts of NoSQL databases. It notes that traditional single-processor relational database management systems (RDBMS) struggled to handle the increasing volume, velocity, variability, and agility of data due to various limitations. This led engineers to explore scaled-out solutions using multiple processors and NoSQL databases, which embrace concepts like horizontal scaling, schema flexibility, and high performance on commodity hardware. Popular NoSQL database models include key-value stores, column-oriented databases, document stores, and graph databases.

Handling Massive WritesLiran Zelkha

This document discusses handling massive writes for online transaction processing (OLTP) systems. It begins with an introduction and overview of the topics to be covered, including terminology, differences between massive reads versus writes, and potential solutions using relational databases, NoSQL databases, and code optimizations. Specific solutions discussed for massive writes include using memory, fast disks, caching, column-oriented databases, SQL tuning, database partitioning, reading from slaves, and sharding or splitting data across multiple databases. The document provides pros and cons of each approach and examples of performance improvements observed.

noSQL choiceslugiamaster4

Cassandra consistently outperforms other NoSQL databases in throughput and scalability according to various benchmark tests, but has higher read latencies. MongoDB typically has the worst performance in terms of latency. The best database depends on application requirements - no single NoSQL database is best for all use cases. Combining database types, such as using Cassandra for analytics and an RDBMS for transactions, can leverage each database's strengths.

Overview of Redundant Disk ArraysAndrew Robinson

Voldemort Nosqlelliando dias

Project Voldemort is a distributed key-value store inspired by Amazon Dynamo and Memcached. It was originally developed at LinkedIn to handle high volumes of data and queries in a scalable way across multiple servers. Voldemort uses consistent hashing to partition and replicate data, vector clocks to resolve concurrent write conflicts, and a layered architecture to provide flexibility. It prioritizes performance, availability, and simplicity over more complex consistency guarantees. LinkedIn uses multiple Voldemort clusters to power various real-time services and applications.

Writing Scalable Software in JavaRuben Badaró

eHarmony in the CloudCraig Dickson

Designing, Building, and Maintaining Large Cubes using Lessons LearnedDenny Lee

Optimize MySQL performance for developersjackbillow

This document provides an overview of optimizing MySQL for developers. It discusses database architecture options like relational and non-relational databases. It also covers caching, scaling through replication and functional segmentation, and optimizing schemas through normalization. The document emphasizes minimizing data access in applications by reducing the number of interactions with the database through techniques like prepared statements. It concludes that architectural design and minimizing data access are key to improving database performance.

Scaling a High Traffic Web Application: Our Journey from Java to PHP120bi

Scaling High Traffic Web ApplicationsAchievers Tech

Архитектура приложений с использованием MySQL, Петр Зайцев (Percona)Ontico

Redis - The Universal NoSQL ToolEberhard Wolff

Webinar: Migrating from RDBMS to MongoDBMongoDB

Make Life Suck Less (Building Scalable Systems)Bradford Stephens

Redis Everywhere - Sunshine PHPRicard Clau

Make Life Suck Less (Building Scalable Systems)guest0f8e278

Rapid Cluster Computing with Apache Spark 2016Zohar Elkayam

Big Data in a Public CloudCloudSigma

Things Every Oracle DBA Needs to Know about the Hadoop EcosystemZohar Elkayam

NoSQLDatabasesAdi Challa

Handling Massive WritesLiran Zelkha

noSQL choiceslugiamaster4

Overview of Redundant Disk ArraysAndrew Robinson

Voldemort Nosqlelliando dias

Writing Scalable Software in JavaRuben Badaró

eHarmony in the CloudCraig Dickson

Designing, Building, and Maintaining Large Cubes using Lessons LearnedDenny Lee

Optimize MySQL performance for developersjackbillow

More from Redis Labs (20)

Redis Day Bangalore 2020 - Session state caching with redisRedis Labs

This document discusses using Redis caching to improve performance for the DBS Paylah mobile wallet application. Paylah aims to significantly increase its user base which will increase load on its backend systems. Caching application data and session state in Redis can reduce latency, improve responsiveness for users, and reduce costs by lowering load on legacy backend databases and mainframes. The document outlines some key Paylah use cases where caching transaction histories and account details in Redis would accelerate retrieval and improve the mobile experience by avoiding the need to access slower backend systems on each request.

Protecting Your API with Redis by Jane Paek - Redis Day Seattle 2020Redis Labs

The document discusses rate limiting and metering using Redis. It begins by introducing rate limiting and metering and why Redis is well-suited for these tasks. It then covers different Redis data structures that can be used, such as lists, hashes, sorted sets and strings. Common Redis commands for counting, setting keys and checking time to live are also presented. Different rate limiting design patterns and anti-patterns are described, including fixed window, sliding window and token bucket approaches. Finally, resources for further information are provided.

The Happy Marriage of Redis and Protobuf by Scott Haines of Twilio - Redis Da...Redis Labs

The document summarizes a presentation about using Protocol Buffers and Redis together. It discusses how Protocol Buffers provide strict data types, versioning, and serialization/deserialization benefits. It then outlines Redis key patterns using namespaces, versions, data categories and identifiers. Examples are provided to show how Protocol Buffers messages can be stored in Redis using these key patterns, including storing connections data in sets, sorted sets and individual messages. Benefits discussed include structure, readability, testing and abstraction.

SQL, Redis and Kubernetes by Paul Stanton of Windocks - Redis Day Seattle 2020Redis Labs

The document discusses common use cases for combining SQL, Redis, and Kubernetes including caching, session management, rate limiting, and data ingestion. It outlines how Kubernetes can be used for scaling microservices while Redis is used for data service scaling. The presentation proposes combining Redis, SQL Server, and Kubernetes with a proxy service, and describes using Redis for caching, session storage, and rate limiting of SQL data. It also discusses running Redis and front-end apps on Kubernetes and deploying SQL as a Kubernetes service through a proxy.

Rust and Redis - Solving Problems for Kubernetes by Ravi Jagannathan of VMwar...Redis Labs

This document discusses using Rust and Redis to build cloud native platforms. It first provides context about devops and the need to do more with less. It then discusses how platforms are becoming more distributed and Kubernetes upends distribution paradigms. The document dives into how Rust addresses issues like concurrency and systems programming. It also discusses how Redis can be used for caching, queues, streams and more. Finally, it mentions that Rust and Redis will be demonstrated.

Redis for Data Science and Engineering by Dmitry Polyakovsky of OracleRedis Labs

This document contains a presentation about using Redis for data science and engineering. It introduces the presenter and provides an agenda that covers using Redis for data science and data engineering. The presentation notes that Redis can be used as both a data store and job queue, has flexible data structures and is fast, though it uses RAM and cannot query by value. It also lists Python Pandas and includes a demo and links for further information.

Practical Use Cases for ACLs in Redis 6 by Jamie Scott - Redis Day Seattle 2020Redis Labs

Jamie Scott from RedisLabs presented on practical use cases for access control lists (ACLs) in Redis 6. The presentation covered new security features in Redis 6 including encryption in transit, key space and command restrictions, and multiple access control list users. It demonstrated how ACLs allow users to define access based on key labels and restrictions. ACLs can facilitate discretionary and mandatory access controls. The presentation showed examples of using ACLs to restrict user access by key labels and commands to enhance operational security.

Moving Beyond Cache by Yiftach Shoolman Redis Labs - Redis Day Seattle 2020Redis Labs

This document summarizes a presentation about Redis version 6 and beyond. Some key points include: - Redis version 6 includes new features like ACL for security, client-side caching, diskless replication, and multi-threaded I/O. - Redis is positioned as both a cache and a database due to its speed, data structures, and ability to handle complex data models through modules. - Redis Enterprise provides additional capabilities like durability, high availability, geo-distribution, security and multi-tenancy. - Modern data models in Redis modules include Streams, RediSearch, RedisGraph, RedisTimeSeries, RedisAI, RedisJSON and RedisBloom. - RedisInsight is

Leveraging Redis for System Monitoring by Adam McCormick of SBG - Redis Day S...Redis Labs

The document discusses how Sinclair Broadcast Group leverages Redis for system monitoring of its content delivery network. It operates 193 news stations with 10,000 active pages daily and millions in archive. New stories are posted every 15 seconds and must be visible across its 1,000+ targets within 1 minute. Redis is used to track performance across the multi-level CDN and ensure service level agreements are met with real-time resolution and alerting. It provides a black box view of the audience experience and can scale monitoring to all relevant pages within 30 seconds. Redis acts as a distributed data store to parallelize the monitoring task across the large scale of the network.

JSON in Redis - When to use RedisJSON by Jay Won of Coupang - Redis Day Seatt...Redis Labs

The document summarizes a presentation about when to use the RedisJSON data type. It discusses how Coupang uses Redis extensively for their ad platform. It then compares the performance and memory usage of storing JSON data as strings, hashes, or using the RedisJSON data type. Benchmark results show RedisJSON can provide better performance for retrieving and updating JSON fields compared to strings and hashes, though it uses more memory. The document recommends using RedisJSON for smaller JSON payloads after benchmarking and memory monitoring.

Highly Available Persistent Session Management Service by Mohamed Elmergawi o...Redis Labs

The document discusses the challenges of building a highly available persistent session management service. It describes Zulily's legacy architecture which lacked high availability and required manual intervention. A new architecture is proposed using Redis for persistent storage, Dynomite for real-time replication across data centers, and a connection pooling proxy to improve efficiency and distribute load. The architecture provides high availability through replication, reduces overhead through connection pooling, and handles failures through consistent hashing and health checks. It was tested through simulations and showed a failure rate of only 0.42% during outages.

Anatomy of a Redis Command by Madelyn Olson of Amazon Web Services - Redis Da...Redis Labs

The document describes the process that a Redis command follows from the client side to the server side. On the client side, the command is sent over the network to the Redis server. On the server side, the command is read from the kernel buffers, validated, executed by calling the relevant command handler, and the response is written back to the client over the network. The core functions involved on the server side are ReadQueryFromClient(), ProcessInputBuffer(), ProcessCommand(), Call(), and handleClientsWithPendingWrites(). Redis 6.0 introduced I/O threads to handle reads and writes in parallel for improved performance while still maintaining Redis' single-threaded processing model.

Building a Multi-dimensional Analytics Engine with RedisGraph by Matthew Goos...Redis Labs

This document discusses MDmetrix, a healthcare data intelligence company that uses RedisGraph to provide flexible analysis of hospital data. RedisGraph is a graph database that represents data as nodes and relationships and uses an adjacency matrix and linear algebra to query the graph. MDmetrix models its healthcare data as a property graph in RedisGraph to allow for complex queries across different data dimensions like patients, facilities, procedures and drugs. RedisGraph allows MDmetrix to query the data more easily than traditional OLAP cubes or relational databases due to the semi-structured and flexible nature of the graph model.

RediSearch 1.6 by Pieter Cailliau - Redis Day Bangalore 2020Redis Labs

RediSearch 1.6 includes a new low-level API that allows other Redis modules to embed RediSearch indexing capabilities. It also introduces index aliasing and several performance improvements such as forked thread garbage collection. Based on benchmarks, RediSearch 1.6 shows 48-73% better performance than version 1.4, particularly during high update rates where it maintains more stable read latencies.

RedisGraph 2.0 by Pieter Cailliau - Redis Day Bangalore 2020Redis Labs

RedisGraph 2.0 provides significant improvements including: - Full text search support through embedded RediSearch 1.6 enabling graph-aided search. - Support for returning full graph responses to enable better visualization. - Broad support for Cypher including triadic closure and new graph-aided search capabilities. - Performance improvements of up to 3.7x faster operations per second and 3.9x faster query times compared to RedisGraph v1.2. - Support for benchmarking including the LDBC benchmark.

RedisTimeSeries 1.2 by Pieter Cailliau - Redis Day Bangalore 2020Redis Labs

RedisTimeSeries is a time-series database that provides compression to reduce memory usage by up to 98% and improve performance. The RedisTimeSeries 1.2 release includes compression algorithms based on a Facebook paper that provide stable ingestion times independent of the number of data points. It also includes a reviewed API with performance improvements and clearer functionality. Performance testing showed ingestion throughput improved by 2-3% and query performance increased from 15-70% with the new release compared to the previous version.

RedisAI 0.9 by Sherin Thomas of Tensorwerk - Redis Day Bangalore 2020Redis Labs

This document summarizes RedisAI 0.9 and its capabilities for model deployment and benchmarking. It introduces RedisAI's new tensor data type and ability to deploy models to CPU and GPU. It then discusses AIBench, a tool developed to benchmark AI serving solutions like RedisAI, TensorFlow Serving, and REST APIs. The benchmarks show RedisAI providing 5.5x and 2.5x more inferences than REST APIs and TensorFlow Serving respectively, due to its data locality. The document concludes by mentioning RedisAI's integration with MLFlow for model deployment with a single command.

Rate-Limiting 30 Million requests by Vijay Lakshminarayanan and Girish Koundi...Redis Labs

The document discusses how Freshworks uses Redis Labs to rate limit 30 million API requests per day through their API gateway called Fluffy. Fluffy stores rate limit policies and maintains counters to track requests. Redis Labs allows Fluffy to easily scale to handle the high volume of requests by providing a fast, in-memory data store for managing rate limiting counters. The system was able to successfully rate limit 30 million requests per day with Redis Labs.

Three Pillars of Observability by Rajalakshmi Raji Srinivasan of Site24x7 Zoh...Redis Labs

Solving Complex Scaling Problems by Prashant Kumar and Abhishek Jain of Myntr...Redis Labs

Redis was used by Myntra to solve several complex scaling problems. It was used to build a scalable user segment service to support high read throughput of up to 5 million requests per minute with low latency. Redis allowed the service to scale beyond a single instance and included features like automatic backups and memory management. Redis also helped build a scalable mobile verification platform to reliably handle 100,000 requests per minute and scale to support higher future volumes. It was used as both a transient store and persistent backend. Finally, Redis locks helped build a scalable A/B testing platform by allowing experiments to be created and updated in an orderly concurrent fashion.