Redis Developers Day 2014 - Redis Labs Talks

Oct 6, 2014Download as PPTX, PDF7 likes2,062 views

These are the slides that the Redis Labs team had used to accompany the session that we gave during the first ever Redis Developers Day on October 2nd, 2014, London. It includes some of the ideas we've come up with to tackle operational challenges in the hyper-dense, multi-tenants Redis deployments that our service - Redis Cloud - consists of.

1st talk: day to day challenges
● ±150 new databases every day (>60,000 total)
● A mix of very small (few MB) and extremely large (TB+) datasets
● PAYG model that can grow to any size by changing maxmemory
● ±250 node failure events (~10 every month) – w/o upgrades
● 5-6 complete data-center outages so far
● Supporting/managing DBs with zero knowledge of application
● Running a modified version of Redis from day 1

2nd talk: Extending Redis to Flash
● Redis #1 hindering factor is the cost of deployment ➔
RAM costs x10 SSD and x250 HDD
● Advanced flash technologies:➔ 1M IOPS (4KB)
➔ Latency: DIMM – 1-2μsec; Plugable – 100μsec

Why do we modify Redis?
● Dense multi-tenant architecture and still maintaining
SLAs (single-tenant, dedicated instances supported)
● Efficient large-scale system administration
● Clustering (in production since 06/2012)

How do we manage changes?
● Many custom modifications are of no use for anyone else, so we
don’t bother
● Fixes, minor changes to upstream ASAP
● Periodic batches of more significant features
o Low hit rate, community view vs. enterprise/service provider
view
● Our “playground” fork https://github.com/redislabs/redis

Skin-of-our-teeth replication
● DB is growing, sync to a bigger instance
● Lots of writes, output buffers and COW bloat
● Redis has a point of no return
⇓
Off-load some of this to the slave
Flush output buffer to slave, Mux RDB

Improved Replication Performance
Master
Memory
Master Disk Network Slave Disk Slave
Memory
● Eliminate disk impact on replication performance (pain point)
● Current Redis scheme can still be better sometimes

Lua Replication
● Old debate: SCRIPT vs Commands
o In some cases performance is dramatic
o We think both options are needed (user-controlled)

External RDB processing
● We do it often, and not only us:
https://github.com/sripathikrishnan/redis-rdb-tools
● Python is too slow for serious work
● We want to avoid code duplication
● Built-in rdb.c was too hard-coded
● Alternative SAX-like (callback-based) parser
● Maybe even librdb.a?

Resource Management
● maxmemory can be confusing (think eviction/OOM because of
output buffers)
● How big can we expect the actual process? Master switches are
missing (e.g. max allocated network buffers)
● Graceful handling of output buffer limits
● Fragmentation issues
o jemalloc and the 12% magic
o active defrag and bigger ideas for bigger problems

More Resource Management
● Automatic process oom_score_adj settings to guard
data in low memory events
● Automatically reject BGSAVE and BGREWRITEAOF or
terminate child on low memory

Cluster features
● Lua key ranges / patterns
● Global keys

Extending Redis to Flash - goals
● More (cheap) memory not better persistence
● 100% compatibility with Redis
● Pluggable storage back-ends / non-volatile DIMM
● Utilize storage concurrency
● Use case: limited set of hot objects over large keyspace
● Different from other approaches: Ledis, ARDB, SSDB

Extending Redis to Flash - blocks
Main thread
Clients
I/O job
queues
I/O
Threads
K/V storage
backend API
Storage
backend

Extending Redis to Flash - moar
● Still being prepared for prime time
● Planned to be open sourced once ready

The document discusses new memory technologies like persistent memory and their implications. It provides latency and bandwidth numbers for different memory types and notes that heterogeneous memory systems using tiers of DRAM and NVM provide opportunities for better performance and cost. Examples are given of key-value stores and databases leveraging NVM to achieve high performance while reducing costs. The talk also discusses how new distributed data structures like CRDTs could be used across servers with shared memory.

Redis for horizontally scaled data processing at jFrog bintrayRedis Labs

Using Redis at FacebookRedis Labs

The document discusses how Redis is used at Facebook to solve two problems: 1) Serving user configurations in real-time by using a hierarchical master-slave replication architecture with global and regional masters and slaves to distribute reads and allow writes at different levels. 2) Processing stats data in real-time by using a split data processing model where daily bulk processing is done separately from real-time processing by tailers that insert stats into a sharded Redis cluster using HINCRBY.

HIgh Performance Redis- Tague Griffith, GoProRedis Labs

High Performance Redis looks at a wide range of techniques - from programming to system tuning - to deploy and maintain an extremely high performing Redis cluster. From the operational perspective, the talk lays out multiple techniques for clustering (sharding) Redis systems and examines how the different approaches impact performance time. The talk further examines system settings (Linux network parameters, Redis system) and how they impact performance (both good and bad). Finally, for the developer, we look at how different ways of structuring data actually demonstrate different performance characteristics

Troubleshooting redisDaeMyung Kang

This document discusses troubleshooting Redis. Some key points: - Redis is single-threaded, so commands like KEYS, FlushAll, and deleting large collections can be slow. It's better to use SCAN instead of KEYS. - Creating Redis database snapshots (RDB files) and rewriting the append-only file (AOF) can cause high disk I/O and CPU usage. It's best to disable automatic rewrites. - Monitoring memory usage and fragmentation is important to avoid performance issues. The maxmemory setting also needs monitoring to prevent out-of-memory errors. - Network and replication failures need solutions like DNS failover or using Zookeeper for coordination to maintain high availability of Redis

Introduction to RedisTO THE NEW | Technology

RedisConf17- Using Redis at scale @ TwitterRedis Labs

The document discusses Nighthawk, Twitter's distributed caching system which uses Redis. It provides caching services at a massive scale of over 10 million queries per second and 10 terabytes of data across 3000 Redis nodes. The key aspects of Nighthawk's architecture that allow it to scale are its use of a client-oblivious proxy layer and cluster manager that can independently scale and rebalance partitions across Redis nodes. It also employs replication between data centers to provide high availability even in the event of node failures. Some challenges discussed are handling "hot keys" that get an unusually high volume of requests and more efficiently warming up replicas when nodes fail.

Managing 50K+ Redis Databases Over 4 Public Clouds ... with a Tiny Devops TeamRedis Labs

Redis is an in-memory database that provides fast performance for powering lightning fast apps. It supports many data structures like strings, hashes, lists, sets and sorted sets. Redis is efficient due to its support for many data structures and commands, as well as its complexity-aware design. Redis Labs provides fully-managed cloud services for Redis and Memcached, and helps customers with challenges around scalability, high availability, performance and monitoring for large-scale Redis deployments.

Counting image views using redis clusterRedis Labs

Streaming Logs and Processing View Counts using Redis Cluster Seandon Mooy (Imgur) When you browse through Imgur, you notice that each user's post includes the number of views for that particular post. Imgur processes over 3 billion views per month and powers our view count feature using Redis. In this talk, we cover our current architecture for streaming logs and processing view counts using Redis Cluster, as well as some of the alternatives we explored and why we chose Redis.

What's new with enterprise Redis - Leena Joshi, Redis LabsRedis Labs

Redis Labs manages over 160k+ HA databases, 10k clustered databases, without data loss in spite of one node failure a day and one data center outage per month. Using Enterprise Redis(RLEC), Redis Labs delivers seamless zero downtime scaling, true high availability with persistence, cross-rack/zone/ datacenter replication and instant automatic failover. Learn how. Join this session for a deep dive into how enterprise Redis makes for no-hassle Redis deployments and the roadmap for new Redis capabilities. Discover new cost savings with Redis on Flash for cost-effective high performance operations and analytics

Redis Labs and SQL ServerLynn Langit

Redis : Database, cache, pub/sub and more at Jelly button gamesRedis Labs

Walmart & IBM Revisit the Linear Road Benchmark- Roger Rea, IBMRedis Labs

The document compares the performance of IBM Streams to other streaming analytics offerings using the Linear Road benchmark, finding that IBM Streams achieved an L-Rating of 200 using 4 Azure nodes, significantly outperforming Apache Apex and Apache Storm. It also describes how Walmart uses streaming analytics for real-time inventory control and logistics monitoring, and how IBM Streams was able to implement the Linear Road benchmark in under 15 days of development time.

RedisConf17 - Home Depot - Turbo charging existing applications with RedisRedis Labs

The Home Depot is transforming its architecture to use microservices and polyglot persistence to handle increasing online order volumes of 250,000 lines per hour. Redis is being used to turbo charge existing monolithic applications by offloading pieces to new processes using patterns like caching, concurrency management, and powering algorithms. This improves performance by reducing database degradation and wait times by over 95%. Next steps include setting up Redis clusters on-premises and off-premises to further reduce database CPU usage and onboard more patterns.

How we got to 1 millisecond latency in 99% under repair, compaction, and flus...ScyllaDB

Scylla is an open source reimplementation of Cassandra which performs up to 10X with drop in-replacement compatibility. At ScyllaDB, performance matters but even more importantly, stable performance under any circumstances. A key factor for our consistent performance is our reliance on userspace schedulers. Scheduling in userspace allows the application, the database in our case to have better control on the different priorities each task has and to provide an SLA to selected operations. Scylla used to have an I/O scheduler and recently won a CPU scheduler. At ScyllaDB, we make architectural decisions that provide not only low latencies but consistently low latencies at higher percentiles. This begins with our choice of language and key architectural decisions such as not using the Linux page-cache, and is fulfilled by autonomous database control, a set of algorithms, which guarantees that the system will adapt to changes in the workload. In the last year, we have made changes to Scylla that provide latencies that are consistent in every percentile. In this talk, Dor Laor will recap those changes and discuss what ScyllaDB is doing in the future.

RedisConf17 - Redis in High Traffic Adtech StackRedis Labs

Rahul Babbar presented on Times Internet's use of Redis in their high-traffic adtech stack. Some key points: - Times Internet is India's largest digital company with 240+ million unique visitors per month. - Their adtech stack powers ads for 150+ publishers and handles 9 billion ad impressions per month. - Redis is used for caching, user profiles, analytics and more due to its low latency and ability to handle high volumes. - Their Redis clusters consist of over 160 nodes across 4 clusters, serving over 2 million queries per second with 99% under 2ms response times.

Using Redis as Distributed Cache for ASP.NET apps - Peter Kellner, 73rd Stre...Redis Labs

I will build from scratch in this session a Microsoft ASP.NET website that caches WebAPI REST calls with both MSOpenTech’s Redis implementation for running while developing in Visual Studio as well as running on a Windows server running IIS. I will show you how to build a safe reusable caching library in c# that can be used in any .net project. I will also demonstrate how to use the Redis cache services that are available on Microsoft’s Azure cloud platform. Further, I’ll demonstrate a real world web site that uses Azure Redis cache and show statistics on how Redis improves performance consistently and reliably.

Scylla Summit 2018: Consensus in Eventually Consistent DatabasesScyllaDB

Eventually consistent databases choose to remain available under failure, allowing for conflicting data to be stored in different replicas (later repaired by background processes). Weakening the consistency guarantees improves not only availability, but also performance, as the number of replicas involved in a given operation can be minimized. There are, however, use-cases that require the opposite trade-off. Indeed, Apache Cassandra and Scylla provide Lightweight Transactions (LWT), which allow single-key linearizable updates. The mechanism underlying LWT is asynchronous consensus. In this talk, we'll describe the characteristics and requirements of Scylla's consensus implementation, and how it enables strongly consistent updates. We will also cover how consensus can be applied to other aspects of the system, such as schema changes, node membership, and range movements, in order to improve their reliability and safety. We will thus show that an eventually consistent database can leverage consensus without compromising either availability or performance.

Perforce BTrees: The Arcane and the ProfanePerforce

DataEngConf SF16 - BYOMQ: Why We [re]Built IronMQHakka Labs

The document discusses reasons for reimplementing messaging queue functionality instead of using an existing solution like RabbitMQ or Kafka. It notes that reimplementing provides full understanding and control, allowing for quick fixes and easy addition of features without needing a daemon. However, it also acknowledges that reimplementing takes a long time to reach stability. The document then describes the specific requirements for the reimplemented messaging queue and provides a brief overview of its design and performance.

RedisConf17 - Lyft - Geospatial at Scale - Daniel HochmanRedis Labs

This document discusses Lyft's migration from a monolithic architecture to using Redis to power a geospatial indexing system at scale. It describes Lyft's original architecture, issues they faced, and how they iterated on their data model and use of Redis over time. It also discusses how Lyft uses Redis across their platform, including operations, monitoring, capacity planning, and their work contributing to open source projects like Envoy.

hbaseconasia2017: Large scale data near-line loading method and architectureHBaseCon

This document proposes a read-write split near-line data loading method and architecture to: - Increase data loading performance by separating write operations from read operations. A WriteServer handles write requests and loads data to HDFS to be read from by RegionServers. - Control resources used by write operations to ensure read operations are not starved of resources like CPU, network, disk I/O, and handlers. - Provide an architecture corresponding to Kafka and HDFS for streaming data from Kafka to HDFS to be loaded into HBase in a delayed manner. - Include optimizations like task balancing across WriteServer slaves, prioritized compaction of small files, and customizable storage engines. - Report test results showing one Write

Managing Redis with Kubernetes - Kelsey Hightower, GoogleRedis Labs

Володимир Цап "Constraint driven infrastructure - scale or tune?"Fwdays

Volodymyr Tsap discusses how to save money on infrastructure through constraint driven design. He provides examples of hardware configurations on AWS, bare metal servers, and PaaS platforms to demonstrate how costs can be optimized. Tsap also outlines ways to reduce software costs through choices in operating system, virtualization, databases, and orchestration. Infrastructure support costs depend on the complexity of the environment, with basic setups costing $500-800 per month while more advanced architectures are $4,000-6,000 per month. The overall message is that money saved through optimization can be invested in people.

Running Analytics at the Speed of Your BusinessRedis Labs

The speed at which you can extract insights from your data is increasingly a competitive edge for your business. Data and analytics have to be at lightning fast speeds to seriously impact your user acquisition. Join this webinar featuring Forrester analyst Noel Yuhanna and Leena Joshi, VP Product Marketing at Redis Labs to learn how you can glean insights faster with new open source data processing frameworks like Spark and Redis. In this webinar you will learn: * Why analytics has to run at the real time speed of business * How this can be achieved with next generation Big Data tools * How data structures can optimize your hybrid transaction-analytics processing scenarios

Scylla Summit 2022: New AWS Instances Perfect for ScyllaDBScyllaDB

In this talk AWS’ Ken Krupa, Head of Specialized Solutions Architecture, will describe the architecture and capabilities of two new AWS EC2 instance types perfect for data-intensive storage and IO-heavy workloads like ScyllaDB: the Intel-based I4i and the Graviton2-based I4g series. The Intel Xeon Ice Lake-based I4i series provides unparalleled raw horsepower for your most demanding workloads. Meanwhile, the Graviton2-powered I4g instances provide lower cost per storage on a power-efficient platform to deploy your cloud-native applications. Ken will also describe the AWS Nitro SSD, a new form of high-speed NVMe storage with a Flash Translation Layer built with Nitro controllers, which powers both of these instance families. ScyllaDB VP of Product Tzach Livyatan will then share benchmarking results showing how ScyllaDB behaves under load on these two instance types, providing maximum system utility and efficiency. To watch all of the recordings hosted during Scylla Summit 2022 visit our website here: https://www.scylladb.com/summit.

Scylla Summit 2018: Keeping Your Latency SLAs No Matter What!ScyllaDB

BDM37: Hadoop in production – the war stories by Nikolaï Grigoriev, Principal...Big Data Montreal

LDAP at Lightning SpeedC4Media

The document discusses LDAP at Lightning Speed and provides information about LMDB, a new key-value database created by Howard Chu that is optimized for LDAP backends. LMDB uses a single-level store design with memory-mapped files and copy-on-write to provide fully transactional, ACID-compliant access without the need for caching, locking, or write-ahead logging. It has a simple configuration and outperforms Berkeley DB while using a fraction of the code size.

More Related Content

What's hot (20)

Managing 50K+ Redis Databases Over 4 Public Clouds ... with a Tiny Devops TeamRedis Labs

Counting image views using redis clusterRedis Labs

What's new with enterprise Redis - Leena Joshi, Redis LabsRedis Labs

Redis Labs and SQL ServerLynn Langit

Redis : Database, cache, pub/sub and more at Jelly button gamesRedis Labs

Walmart & IBM Revisit the Linear Road Benchmark- Roger Rea, IBMRedis Labs

RedisConf17 - Home Depot - Turbo charging existing applications with RedisRedis Labs

How we got to 1 millisecond latency in 99% under repair, compaction, and flus...ScyllaDB

RedisConf17 - Redis in High Traffic Adtech StackRedis Labs

Using Redis as Distributed Cache for ASP.NET apps - Peter Kellner, 73rd Stre...Redis Labs

Scylla Summit 2018: Consensus in Eventually Consistent DatabasesScyllaDB

Perforce BTrees: The Arcane and the ProfanePerforce

DataEngConf SF16 - BYOMQ: Why We [re]Built IronMQHakka Labs

RedisConf17 - Lyft - Geospatial at Scale - Daniel HochmanRedis Labs

hbaseconasia2017: Large scale data near-line loading method and architectureHBaseCon

Managing Redis with Kubernetes - Kelsey Hightower, GoogleRedis Labs

Володимир Цап "Constraint driven infrastructure - scale or tune?"Fwdays

Running Analytics at the Speed of Your BusinessRedis Labs

Scylla Summit 2022: New AWS Instances Perfect for ScyllaDBScyllaDB

Scylla Summit 2018: Keeping Your Latency SLAs No Matter What!ScyllaDB

Managing 50K+ Redis Databases Over 4 Public Clouds ... with a Tiny Devops TeamRedis Labs

Counting image views using redis clusterRedis Labs

What's new with enterprise Redis - Leena Joshi, Redis LabsRedis Labs

Redis Labs and SQL ServerLynn Langit

Redis : Database, cache, pub/sub and more at Jelly button gamesRedis Labs

Walmart & IBM Revisit the Linear Road Benchmark- Roger Rea, IBMRedis Labs

RedisConf17 - Home Depot - Turbo charging existing applications with RedisRedis Labs

How we got to 1 millisecond latency in 99% under repair, compaction, and flus...ScyllaDB

RedisConf17 - Redis in High Traffic Adtech StackRedis Labs

Using Redis as Distributed Cache for ASP.NET apps - Peter Kellner, 73rd Stre...Redis Labs

Scylla Summit 2018: Consensus in Eventually Consistent DatabasesScyllaDB

Perforce BTrees: The Arcane and the ProfanePerforce

DataEngConf SF16 - BYOMQ: Why We [re]Built IronMQHakka Labs

RedisConf17 - Lyft - Geospatial at Scale - Daniel HochmanRedis Labs

hbaseconasia2017: Large scale data near-line loading method and architectureHBaseCon

Managing Redis with Kubernetes - Kelsey Hightower, GoogleRedis Labs

Володимир Цап "Constraint driven infrastructure - scale or tune?"Fwdays

Running Analytics at the Speed of Your BusinessRedis Labs

Scylla Summit 2022: New AWS Instances Perfect for ScyllaDBScyllaDB

Scylla Summit 2018: Keeping Your Latency SLAs No Matter What!ScyllaDB

Similar to Redis Developers Day 2014 - Redis Labs Talks (20)

BDM37: Hadoop in production – the war stories by Nikolaï Grigoriev, Principal...Big Data Montreal

LDAP at Lightning SpeedC4Media

Scaling AEM (CQ5) Gem SessionMichael Marth

This document discusses various patterns for horizontally scaling an AEM implementation. It begins by defining performance and scalability, and notes that pre-Oak scalability patterns are covered. Eight common use cases for scaling AEM are then described and solutions proposed: 1) high volume delivery, 2) high frequency input, 3) high processing input, 4) high volume input, 5) many editors, 6) geo-distributed editors, 7) many DAM assets, and 8) geo-distributed disaster recovery. For each use case, one or more solution patterns are outlined in one to three sentences.

Scaling CQ5connectwebex

The document discusses various patterns for scaling a CQ5 implementation. It describes scaling patterns for high volume delivery, high frequency input feeds, many editors, high processing input feeds, high volume input feeds, geo-distributed editors, many DAM assets, and geo-distributed disaster recovery. For each pattern, it provides descriptions of use cases and limiting factors, and proposes solutions such as sharding, separating processing instances, batching operations, and leveraging caching and compression.

Evolution of DBA in the Cloud EraMydbops

Cloud arch patternsCorey Huinker

This document discusses PostgreSQL database architecture patterns for running PostgreSQL at scale when a relational database as a service like Amazon RDS won't meet needs. It describes challenges faced with MySQL, Redshift and Vertica and how PostgreSQL was better suited through techniques like partitioning by date, TOAST compression, foreign data wrappers, and poor man's parallel processing. Key takeaways are that PostgreSQL supported scaling to petabytes of data, sub-second queries across large date ranges, and custom extensions needed while avoiding limitations and expenses of other database options.

Ceph: A decade in the making and still going strongPatrick McGarry

Ceph is an open source distributed storage system that has been in development for over a decade. It started as a research project at UC Santa Cruz to build scalable object storage. Over the years, it has grown to include distributed block storage, file storage and an S3-compatible object store. Ceph is now used in many production deployments and has a thriving developer community, though continued work is needed to improve areas like CephFS and add new features around erasure coding, tiering and replication. The future of Ceph involves strengthening governance, expanding the ecosystem, improving performance and gaining more adoption in enterprise storage environments.

Running Production CDC Ingestion Pipelines With Balaji Varadarajan and Pritam...HostedbyConfluent

Running Production CDC Ingestion Pipelines With Balaji Varadarajan and Pritam K Dey | Current 2022 Robinhood’s mission is to democratize finance for all. Data driven decision making is key to achieving this goal. Data needed are hosted in various OLTP databases. Replicating this data near real time in a reliable fashion to data lakehouse powers many critical use cases for the company. In Robinhood, CDC is not only used for ingestion to data-lake but is also being adopted for inter-system message exchanges between different online micro services. . In this talk, we will describe the evolution of change data capture based ingestion in Robinhood not only in terms of the scale of data stored and queries made, but also the use cases that it supports. We will go in-depth into the CDC architecture built around our Kafka ecosystem using open source system Debezium and Apache Hudi. We will cover online inter-system message exchange use-cases along with our experience running this service at scale in Robinhood along with lessons learned.

Monitoring with Clickhouseunicast

- Clickhouse is being used by GoEuro to replace their Graphite backend for monitoring, as their previous Graphite setup required too much maintenance and tuning over time to handle their scale of 20 million visitors per month, 150 engineers, and 600+ releases per week. - Key reasons for choosing Clickhouse include its built-in replication, sharding, linear scalability, and a GraphiteMergeTree table engine that provides 100% compatibility with the Graphite query language. - Downsides of Clickhouse include initial dependency on Zookeeper for sharding/replication and slower read queries against sharded data, but it currently uses only 2 CPU cores and 2GB RAM to handle GoEuro's monitoring needs.

Scalr: Setting Up Automated ScalingHakka Labs

Despite all the buzz about it, building a horizontally scalable application for cloud deployment isn't all that different from building one for a physical deployment, except in its ability to change size on-the-fly. Bigger applications have been using commodity hardware and fault-tolerant design to achieve high availability and scalability for a while, but provisioning capacity remains troublesome there. The real addition the cloud brings architecturally is the ability to add new resources instantly, and even change your provisioning profile algorithmically.

Linux Huge PagesGeraldo Netto

This document discusses Linux huge pages, including: - What huge pages are and how they can reduce memory management overhead by allocating larger blocks of memory - How to configure huge pages on Linux, including installing required packages, mounting the huge page filesystem, and setting kernel parameters - When huge pages should be configured, such as for data-intensive or latency-sensitive applications like databases, but that testing is required due to disadvantages like reduced swappability

Ceph Community Talk on High-Performance Solid Sate Ceph Ceph Community

The document summarizes a presentation given by representatives from various companies on optimizing Ceph for high-performance solid state drives. It discusses testing a real workload on a Ceph cluster with 50 SSD nodes that achieved over 280,000 read and write IOPS. Areas for further optimization were identified, such as reducing latency spikes and improving single-threaded performance. Various companies then described their contributions to Ceph performance, such as Intel providing hardware for testing and Samsung discussing SSD interface improvements.

Retour d'expérience d'un environnement base de données multitenantSwiss Data Forum Swiss Data Forum

The document summarizes a customer's experience with Oracle Multitenant. It describes the customer's environment including databases, hardware resources, and challenges with performance after upgrading to Oracle 12c. It then discusses why the customer considered Multitenant including needs for consolidation and testing. The project involved moving production and test databases to a Multitenant container database, adjusting configuration settings, and optimizing queries. The results were improved performance and ability to scale resources. New features in Oracle 12.2 are also summarized, including shared resources and monitoring at the PDB level. Human: Thank you for the summary. Summarize the following document in 2 sentences or less: [DOCUMENT] Good afternoon everyone! Thank you for

Scaling up and accelerating Drupal 8 with NoSQLOSInet

Drupal 7 performance and optimizationShafqat Hussain

Running MySQL in AWSLaine Campbell

This document discusses options for running MySQL in AWS. It describes using Amazon RDS, where AWS manages the infrastructure and MySQL version, but has limitations like lack of root access. It also describes using EC2, where one provisions and manages their own instances, storage, and MySQL binaries, allowing more flexibility but also more management overhead. Key tradeoffs discussed are ease of use vs customization options and control in RDS vs EC2.

Using ScyllaDB for Real-Time Read-Heavy Workloads.pdfScyllaDB

Keeping latencies predictably low for instant user experiences ScyllaDB’s “sweet spot” is workloads over 50K operations per second that require predictably low (e.g., single-digit millisecond) latency. And its unique architecture makes it particularly valuable for the real-time read-heavy workloads such as those commonly found in recommendation, personalization, catalog, gaming, chat, and similar use cases. Join ScyllaDB technical director Felipe Cardeneti Mendes and senior director of product experience Tim Koopmans to learn about: - Common challenges that arise with real-time read-heavy workloads - The tradeoffs teams face and tips for negotiating them - ScyllaDB architectural elements that support real-time read-heavy workloads - How your peers are using ScyllaDB with similar workloads

kranonit S06E01 Игорь Цинько: High loadKrivoy Rog IT Community

This document summarizes a presentation about designing systems to handle high loads when Chuck Norris is your customer. It discusses scaling architectures vertically and horizontally, RESTful principles, using NoSQL databases like MongoDB, caching with Memcached, search engines like Sphinx, video/image storage, and bandwidth management. It emphasizes that the right technology depends on business needs, and high-load systems require robust architectures, qualified developers, and avoiding single points of failure.

Ceph Day New York: Ceph: one decade inCeph Community

Ceph is an open-source distributed storage system that began as a research project in 2005. It has grown significantly since then, with the founding of Inktank in 2012 to support Ceph's development and commercial adoption. Key developments include the native Linux kernel client, erasure coding support, asynchronous replication, and improvements to CephFS. The future of Ceph includes stronger governance, a focus on performance and low-power architectures, integration with big data workloads, and use as an archival storage solution.

PostgreSQL and Redis - talk at pgcon 2013Andrew Dunstan

This document discusses using PostgreSQL and Redis together for a high performance ad serving system. Redis is used as a fast in-memory database to store frequently accessed data like ad impressions and clicks. A Redis foreign data wrapper allows PostgreSQL to efficiently retrieve this data from Redis and load it into the recording database. The PostgreSQL databases store longer term data and handle reporting. Materialized views are used to optimize ad filtering queries. Together PostgreSQL and Redis allow the system to serve over 10,000 ads per second while recording analytics data for reporting.

BDM37: Hadoop in production – the war stories by Nikolaï Grigoriev, Principal...Big Data Montreal

LDAP at Lightning SpeedC4Media

Scaling AEM (CQ5) Gem SessionMichael Marth

Scaling CQ5connectwebex

Evolution of DBA in the Cloud EraMydbops

Cloud arch patternsCorey Huinker

Ceph: A decade in the making and still going strongPatrick McGarry

Running Production CDC Ingestion Pipelines With Balaji Varadarajan and Pritam...HostedbyConfluent

Monitoring with Clickhouseunicast

Scalr: Setting Up Automated ScalingHakka Labs

Linux Huge PagesGeraldo Netto

Ceph Community Talk on High-Performance Solid Sate Ceph Ceph Community

Retour d'expérience d'un environnement base de données multitenantSwiss Data Forum Swiss Data Forum

Scaling up and accelerating Drupal 8 with NoSQLOSInet

Drupal 7 performance and optimizationShafqat Hussain

Running MySQL in AWSLaine Campbell

Using ScyllaDB for Real-Time Read-Heavy Workloads.pdfScyllaDB

kranonit S06E01 Игорь Цинько: High loadKrivoy Rog IT Community

Ceph Day New York: Ceph: one decade inCeph Community

PostgreSQL and Redis - talk at pgcon 2013Andrew Dunstan

More from Redis Labs (20)

Redis Day Bangalore 2020 - Session state caching with redisRedis Labs

This document discusses using Redis caching to improve performance for the DBS Paylah mobile wallet application. Paylah aims to significantly increase its user base which will increase load on its backend systems. Caching application data and session state in Redis can reduce latency, improve responsiveness for users, and reduce costs by lowering load on legacy backend databases and mainframes. The document outlines some key Paylah use cases where caching transaction histories and account details in Redis would accelerate retrieval and improve the mobile experience by avoiding the need to access slower backend systems on each request.

Protecting Your API with Redis by Jane Paek - Redis Day Seattle 2020Redis Labs

The document discusses rate limiting and metering using Redis. It begins by introducing rate limiting and metering and why Redis is well-suited for these tasks. It then covers different Redis data structures that can be used, such as lists, hashes, sorted sets and strings. Common Redis commands for counting, setting keys and checking time to live are also presented. Different rate limiting design patterns and anti-patterns are described, including fixed window, sliding window and token bucket approaches. Finally, resources for further information are provided.

The Happy Marriage of Redis and Protobuf by Scott Haines of Twilio - Redis Da...Redis Labs

The document summarizes a presentation about using Protocol Buffers and Redis together. It discusses how Protocol Buffers provide strict data types, versioning, and serialization/deserialization benefits. It then outlines Redis key patterns using namespaces, versions, data categories and identifiers. Examples are provided to show how Protocol Buffers messages can be stored in Redis using these key patterns, including storing connections data in sets, sorted sets and individual messages. Benefits discussed include structure, readability, testing and abstraction.

SQL, Redis and Kubernetes by Paul Stanton of Windocks - Redis Day Seattle 2020Redis Labs

The document discusses common use cases for combining SQL, Redis, and Kubernetes including caching, session management, rate limiting, and data ingestion. It outlines how Kubernetes can be used for scaling microservices while Redis is used for data service scaling. The presentation proposes combining Redis, SQL Server, and Kubernetes with a proxy service, and describes using Redis for caching, session storage, and rate limiting of SQL data. It also discusses running Redis and front-end apps on Kubernetes and deploying SQL as a Kubernetes service through a proxy.

Rust and Redis - Solving Problems for Kubernetes by Ravi Jagannathan of VMwar...Redis Labs

This document discusses using Rust and Redis to build cloud native platforms. It first provides context about devops and the need to do more with less. It then discusses how platforms are becoming more distributed and Kubernetes upends distribution paradigms. The document dives into how Rust addresses issues like concurrency and systems programming. It also discusses how Redis can be used for caching, queues, streams and more. Finally, it mentions that Rust and Redis will be demonstrated.

Redis for Data Science and Engineering by Dmitry Polyakovsky of OracleRedis Labs

This document contains a presentation about using Redis for data science and engineering. It introduces the presenter and provides an agenda that covers using Redis for data science and data engineering. The presentation notes that Redis can be used as both a data store and job queue, has flexible data structures and is fast, though it uses RAM and cannot query by value. It also lists Python Pandas and includes a demo and links for further information.

Practical Use Cases for ACLs in Redis 6 by Jamie Scott - Redis Day Seattle 2020Redis Labs

Jamie Scott from RedisLabs presented on practical use cases for access control lists (ACLs) in Redis 6. The presentation covered new security features in Redis 6 including encryption in transit, key space and command restrictions, and multiple access control list users. It demonstrated how ACLs allow users to define access based on key labels and restrictions. ACLs can facilitate discretionary and mandatory access controls. The presentation showed examples of using ACLs to restrict user access by key labels and commands to enhance operational security.

Moving Beyond Cache by Yiftach Shoolman Redis Labs - Redis Day Seattle 2020Redis Labs

This document summarizes a presentation about Redis version 6 and beyond. Some key points include: - Redis version 6 includes new features like ACL for security, client-side caching, diskless replication, and multi-threaded I/O. - Redis is positioned as both a cache and a database due to its speed, data structures, and ability to handle complex data models through modules. - Redis Enterprise provides additional capabilities like durability, high availability, geo-distribution, security and multi-tenancy. - Modern data models in Redis modules include Streams, RediSearch, RedisGraph, RedisTimeSeries, RedisAI, RedisJSON and RedisBloom. - RedisInsight is

Leveraging Redis for System Monitoring by Adam McCormick of SBG - Redis Day S...Redis Labs

The document discusses how Sinclair Broadcast Group leverages Redis for system monitoring of its content delivery network. It operates 193 news stations with 10,000 active pages daily and millions in archive. New stories are posted every 15 seconds and must be visible across its 1,000+ targets within 1 minute. Redis is used to track performance across the multi-level CDN and ensure service level agreements are met with real-time resolution and alerting. It provides a black box view of the audience experience and can scale monitoring to all relevant pages within 30 seconds. Redis acts as a distributed data store to parallelize the monitoring task across the large scale of the network.

JSON in Redis - When to use RedisJSON by Jay Won of Coupang - Redis Day Seatt...Redis Labs

The document summarizes a presentation about when to use the RedisJSON data type. It discusses how Coupang uses Redis extensively for their ad platform. It then compares the performance and memory usage of storing JSON data as strings, hashes, or using the RedisJSON data type. Benchmark results show RedisJSON can provide better performance for retrieving and updating JSON fields compared to strings and hashes, though it uses more memory. The document recommends using RedisJSON for smaller JSON payloads after benchmarking and memory monitoring.

Highly Available Persistent Session Management Service by Mohamed Elmergawi o...Redis Labs

The document discusses the challenges of building a highly available persistent session management service. It describes Zulily's legacy architecture which lacked high availability and required manual intervention. A new architecture is proposed using Redis for persistent storage, Dynomite for real-time replication across data centers, and a connection pooling proxy to improve efficiency and distribute load. The architecture provides high availability through replication, reduces overhead through connection pooling, and handles failures through consistent hashing and health checks. It was tested through simulations and showed a failure rate of only 0.42% during outages.

Anatomy of a Redis Command by Madelyn Olson of Amazon Web Services - Redis Da...Redis Labs

The document describes the process that a Redis command follows from the client side to the server side. On the client side, the command is sent over the network to the Redis server. On the server side, the command is read from the kernel buffers, validated, executed by calling the relevant command handler, and the response is written back to the client over the network. The core functions involved on the server side are ReadQueryFromClient(), ProcessInputBuffer(), ProcessCommand(), Call(), and handleClientsWithPendingWrites(). Redis 6.0 introduced I/O threads to handle reads and writes in parallel for improved performance while still maintaining Redis' single-threaded processing model.

Building a Multi-dimensional Analytics Engine with RedisGraph by Matthew Goos...Redis Labs

This document discusses MDmetrix, a healthcare data intelligence company that uses RedisGraph to provide flexible analysis of hospital data. RedisGraph is a graph database that represents data as nodes and relationships and uses an adjacency matrix and linear algebra to query the graph. MDmetrix models its healthcare data as a property graph in RedisGraph to allow for complex queries across different data dimensions like patients, facilities, procedures and drugs. RedisGraph allows MDmetrix to query the data more easily than traditional OLAP cubes or relational databases due to the semi-structured and flexible nature of the graph model.

RediSearch 1.6 by Pieter Cailliau - Redis Day Bangalore 2020Redis Labs

RediSearch 1.6 includes a new low-level API that allows other Redis modules to embed RediSearch indexing capabilities. It also introduces index aliasing and several performance improvements such as forked thread garbage collection. Based on benchmarks, RediSearch 1.6 shows 48-73% better performance than version 1.4, particularly during high update rates where it maintains more stable read latencies.

RedisGraph 2.0 by Pieter Cailliau - Redis Day Bangalore 2020Redis Labs

RedisGraph 2.0 provides significant improvements including: - Full text search support through embedded RediSearch 1.6 enabling graph-aided search. - Support for returning full graph responses to enable better visualization. - Broad support for Cypher including triadic closure and new graph-aided search capabilities. - Performance improvements of up to 3.7x faster operations per second and 3.9x faster query times compared to RedisGraph v1.2. - Support for benchmarking including the LDBC benchmark.

RedisTimeSeries 1.2 by Pieter Cailliau - Redis Day Bangalore 2020Redis Labs

RedisTimeSeries is a time-series database that provides compression to reduce memory usage by up to 98% and improve performance. The RedisTimeSeries 1.2 release includes compression algorithms based on a Facebook paper that provide stable ingestion times independent of the number of data points. It also includes a reviewed API with performance improvements and clearer functionality. Performance testing showed ingestion throughput improved by 2-3% and query performance increased from 15-70% with the new release compared to the previous version.

RedisAI 0.9 by Sherin Thomas of Tensorwerk - Redis Day Bangalore 2020Redis Labs

This document summarizes RedisAI 0.9 and its capabilities for model deployment and benchmarking. It introduces RedisAI's new tensor data type and ability to deploy models to CPU and GPU. It then discusses AIBench, a tool developed to benchmark AI serving solutions like RedisAI, TensorFlow Serving, and REST APIs. The benchmarks show RedisAI providing 5.5x and 2.5x more inferences than REST APIs and TensorFlow Serving respectively, due to its data locality. The document concludes by mentioning RedisAI's integration with MLFlow for model deployment with a single command.

Rate-Limiting 30 Million requests by Vijay Lakshminarayanan and Girish Koundi...Redis Labs

The document discusses how Freshworks uses Redis Labs to rate limit 30 million API requests per day through their API gateway called Fluffy. Fluffy stores rate limit policies and maintains counters to track requests. Redis Labs allows Fluffy to easily scale to handle the high volume of requests by providing a fast, in-memory data store for managing rate limiting counters. The system was able to successfully rate limit 30 million requests per day with Redis Labs.

Three Pillars of Observability by Rajalakshmi Raji Srinivasan of Site24x7 Zoh...Redis Labs

Solving Complex Scaling Problems by Prashant Kumar and Abhishek Jain of Myntr...Redis Labs

Redis was used by Myntra to solve several complex scaling problems. It was used to build a scalable user segment service to support high read throughput of up to 5 million requests per minute with low latency. Redis allowed the service to scale beyond a single instance and included features like automatic backups and memory management. Redis also helped build a scalable mobile verification platform to reliably handle 100,000 requests per minute and scale to support higher future volumes. It was used as both a transient store and persistent backend. Finally, Redis locks helped build a scalable A/B testing platform by allowing experiments to be created and updated in an orderly concurrent fashion.