Redis' seventh birthday was recently celebrated with the community, several contributors and users. This is Salvatore's keynote as he kicked off Redis Day in Tel Aviv.
Redis Developers Day 2014 - Redis Labs TalksRedis Labs
ย
These are the slides that the Redis Labs team had used to accompany the session that we gave during the first ever Redis Developers Day on October 2nd, 2014, London. It includes some of the ideas we've come up with to tackle operational challenges in the hyper-dense, multi-tenants Redis deployments that our service - Redis Cloud - consists of.
Day 2 General Session Presentations RedisConfRedis Labs
ย
The document discusses new memory technologies like persistent memory and their implications. It provides latency and bandwidth numbers for different memory types and notes that heterogeneous memory systems using tiers of DRAM and NVM provide opportunities for better performance and cost. Examples are given of key-value stores and databases leveraging NVM to achieve high performance while reducing costs. The talk also discusses how new distributed data structures like CRDTs could be used across servers with shared memory.
This document discusses troubleshooting Redis. Some key points:
- Redis is single-threaded, so commands like KEYS, FlushAll, and deleting large collections can be slow. It's better to use SCAN instead of KEYS.
- Creating Redis database snapshots (RDB files) and rewriting the append-only file (AOF) can cause high disk I/O and CPU usage. It's best to disable automatic rewrites.
- Monitoring memory usage and fragmentation is important to avoid performance issues. The maxmemory setting also needs monitoring to prevent out-of-memory errors.
- Network and replication failures need solutions like DNS failover or using Zookeeper for coordination to maintain high availability of Redis
HIgh Performance Redis- Tague Griffith, GoProRedis Labs
ย
High Performance Redis looks at a wide range of techniques - from programming to system tuning - to deploy and maintain an extremely high performing Redis cluster. From the operational
perspective, the talk lays out multiple techniques for clustering (sharding) Redis systems and examines how the different
approaches impact performance time. The talk further examines system settings (Linux network parameters, Redis
system) and how they impact performance (both good and bad). Finally, for the developer, we look at how different ways of structuring data actually demonstrate different performance characteristics
RedisConf17- Using Redis at scale @ TwitterRedis Labs
ย
The document discusses Nighthawk, Twitter's distributed caching system which uses Redis. It provides caching services at a massive scale of over 10 million queries per second and 10 terabytes of data across 3000 Redis nodes. The key aspects of Nighthawk's architecture that allow it to scale are its use of a client-oblivious proxy layer and cluster manager that can independently scale and rebalance partitions across Redis nodes. It also employs replication between data centers to provide high availability even in the event of node failures. Some challenges discussed are handling "hot keys" that get an unusually high volume of requests and more efficiently warming up replicas when nodes fail.
Timely genome analysis requires a fresh approach to platform design for big data problems. Louisiana State University has tested enterprise cluster deployments of Redis with a unique solution that allows flash memory to act as extended RAM. Learn about how this solution allows large amounts of data to be handled with a fraction of the memory needed for a typical deployment.
Redis for horizontally scaled data processing at jFrog bintrayRedis Labs
ย
This document discusses using Redis in a horizontally scaled data processing architecture. Redis is used to store data across multiple databases and is accessed through Twemproxy, which acts as a load balancer. Data is sharded across the Redis databases and Twemproxy ensures there is no single point of failure. Events and tasks are queued in Redis and processed asynchronously by microservices.
Counting image views using redis clusterRedis Labs
ย
Streaming Logs and Processing View Counts using Redis Cluster
Seandon Mooy
(Imgur)
When you browse through Imgur, you notice that each user's post includes the number of views for that particular post. Imgur processes over 3 billion views per month and powers our view count feature using Redis. In this talk, we cover our current architecture for streaming logs and processing view counts using Redis Cluster, as well as some of the alternatives we explored and why we chose Redis.
Managing 50K+ Redis Databases Over 4 Public Clouds ... with a Tiny Devops TeamRedis Labs
ย
Redis is an in-memory database that provides fast performance for powering lightning fast apps. It supports many data structures like strings, hashes, lists, sets and sorted sets. Redis is efficient due to its support for many data structures and commands, as well as its complexity-aware design. Redis Labs provides fully-managed cloud services for Redis and Memcached, and helps customers with challenges around scalability, high availability, performance and monitoring for large-scale Redis deployments.
The document discusses how Redis is used at Facebook to solve two problems:
1) Serving user configurations in real-time by using a hierarchical master-slave replication architecture with global and regional masters and slaves to distribute reads and allow writes at different levels.
2) Processing stats data in real-time by using a split data processing model where daily bulk processing is done separately from real-time processing by tailers that insert stats into a sharded Redis cluster using HINCRBY.
How we got to 1 millisecond latency in 99% under repair, compaction, and flus...ScyllaDB
ย
Scylla is an open source reimplementation of Cassandra which performs up to 10X with drop in-replacement compatibility. At ScyllaDB, performance matters but even more importantly, stable performance under any circumstances.
A key factor for our consistent performance is our reliance on userspace schedulers. Scheduling in userspace allows the application, the database in our case to have better control on the different priorities each task has and to provide an SLA to selected operations. Scylla used to have an I/O scheduler and recently won a CPU scheduler.
At ScyllaDB, we make architectural decisions that provide not only low latencies but consistently low latencies at higher percentiles. This begins with our choice of language and key architectural decisions such as not using the Linux page-cache, and is fulfilled by autonomous database control, a set of algorithms, which guarantees that the system will adapt to changes in the workload. In the last year, we have made changes to Scylla that provide latencies that are consistent in every percentile. In this talk, Dor Laor will recap those changes and discuss what ScyllaDB is doing in the future.
Everything you wanted to know about RadosGW - Orit Wasserman, Matt BenjaminCeph Community
ย
This document provides an overview of object storage concepts and features of Ceph's RADOS Gateway (RGW). Key points include:
- RGW provides a RESTful API for object storage that is compatible with AWS S3 and OpenStack Swift.
- Objects stored in RGW can be large in size and immutable, with permissions set at the object level.
- Multipart uploads allow efficient transfer of large objects by splitting them into parts.
- Versioning and lifecycle policies allow automatic management of object versions and transitions.
- RGW can be used to provide NFS access to the object storage namespace.
Red Hat's storage roadmap aims to offer a unified, open software-defined storage portfolio for next generation workloads. Their portfolio includes Ceph and Gluster storage, which provide data services like object storage, block storage, and file sharing on standard hardware. Red Hat is working to improve management, performance, security, and integration of these products. They are also exploring new workloads like running MySQL on Ceph block storage and using Gluster Storage for hyperconverged Red Hat Enterprise Virtualization environments.
DataEngConf SF16 - BYOMQ: Why We [re]Built IronMQHakka Labs
ย
The document discusses reasons for reimplementing messaging queue functionality instead of using an existing solution like RabbitMQ or Kafka. It notes that reimplementing provides full understanding and control, allowing for quick fixes and easy addition of features without needing a daemon. However, it also acknowledges that reimplementing takes a long time to reach stability. The document then describes the specific requirements for the reimplemented messaging queue and provides a brief overview of its design and performance.
Scylla Summit 2018: Consensus in Eventually Consistent DatabasesScyllaDB
ย
Eventually consistent databases choose to remain available under failure, allowing for conflicting data to be stored in different replicas (later repaired by background processes). Weakening the consistency guarantees improves not only availability, but also performance, as the number of replicas involved in a given operation can be minimized. There are, however, use-cases that require the opposite trade-off. Indeed, Apache Cassandra and Scylla provide Lightweight Transactions (LWT), which allow single-key linearizable updates. The mechanism underlying LWT is asynchronous consensus. In this talk, we'll describe the characteristics and requirements of Scylla's consensus implementation, and how it enables strongly consistent updates. We will also cover how consensus can be applied to other aspects of the system, such as schema changes, node membership, and range movements, in order to improve their reliability and safety. We will thus show that an eventually consistent database can leverage consensus without compromising either availability or performance.
RedisConf17 - Redis in High Traffic Adtech StackRedis Labs
ย
Rahul Babbar presented on Times Internet's use of Redis in their high-traffic adtech stack. Some key points:
- Times Internet is India's largest digital company with 240+ million unique visitors per month.
- Their adtech stack powers ads for 150+ publishers and handles 9 billion ad impressions per month.
- Redis is used for caching, user profiles, analytics and more due to its low latency and ability to handle high volumes.
- Their Redis clusters consist of over 160 nodes across 4 clusters, serving over 2 million queries per second with 99% under 2ms response times.
What's new with enterprise Redis - Leena Joshi, Redis LabsRedis Labs
ย
Redis Labs manages over 160k+ HA databases, 10k clustered databases, without data loss in spite of one node failure a day and one data center outage per month. Using Enterprise
Redis(RLEC), Redis Labs delivers seamless zero downtime scaling, true high availability with persistence, cross-rack/zone/
datacenter replication and instant automatic failover. Learn how. Join this session for a deep dive into how enterprise Redis makes for no-hassle Redis deployments and the roadmap for new Redis capabilities. Discover new cost savings with Redis on Flash for cost-effective high performance operations and analytics
Experiences building a distributed shared log on RADOS - Noah WatkinsCeph Community
ย
This document summarizes Noah Watkins' presentation on building a distributed shared log using Ceph. The key points are:
1) Noah discusses how shared logs are challenging to scale due to the need to funnel all writes through a total ordering engine. This bottlenecks performance.
2) CORFU is introduced as a shared log design that decouples I/O from ordering by striping the log across flash devices and using a sequencer to assign positions.
3) Noah then explains how the components of CORFU can be mapped onto Ceph, using RADOS object classes, librados, and striping policies to implement the shared log without requiring custom hardware interfaces.
4) ZLog is presented
Linux Block Cache Practice on Ceph BlueStore - Junxin ZhangCeph Community
ย
This document discusses using Linux block caching with Ceph BlueStore. It explains that BlueStore can better utilize fast storage devices like SSDs compared to FileStore. It tested using Bcache and DM-writeboost to cache BlueStore data on HDDs using SSDs. Bcache performed better overall. Issues found were slow requests when caching and BlueStore used the same SSD, and inconsistency in SSD data management between BlueStore and the cache. Future work could have BlueStore control all raw disks and prioritize data saving to fast devices.
RADOS improvements and roadmap - Greg Farnum, Josh Durgin, Kefu ChaiCeph Community
ย
Cephalocon APAC 2018
March 22-23, 2018 - Beijing, China
Greg Farnum๏ผRed Hat RADOS Core Developer
Josh Durgin, Red Hat RADOS Lead
Kefu Chai, Red Hat Senior Software Engineer
Petabyte Scale Object Storage Service Using Ceph in A Private Cloud - Varada ...Ceph Community
ย
This document discusses Flipkart's use of Ceph object storage to provide a petabyte-scale object storage service. Key points:
- Flipkart runs two large data centers hosting over 20,000 servers and 60,000 VMs to power its e-commerce marketplace.
- It developed a highly scalable object storage service using Ceph to store over 1.5 billion objects totaling around 2PB of data. This service provides APIs compatible with AWS S3.
- The Ceph clusters are deployed across SSDs and HDDs to provide different performance and cost tiers for shared active, backup, and archival workloads with different SLAs around latency, throughput, and durability
Kafka Summit SF 2017 - Shopify Flash Sales with Apache Kafkaconfluent
ย
This document discusses how Shopify uses Apache Kafka in their systems architecture. It describes how Kafka provides reliable asynchronous messaging that allows Shopify to collect logs and events and feed them to their data lake. It outlines how Kafka provides operational decoupling and allows them to deploy application or Kafka changes independently. It then discusses two specific use cases: using Kafka as part of a logs/events pipeline and using it to enable active-active Elasticsearch replication across multiple data centers.
RedisConf17 - Lyft - Geospatial at Scale - Daniel HochmanRedis Labs
ย
This document discusses Lyft's migration from a monolithic architecture to using Redis to power a geospatial indexing system at scale. It describes Lyft's original architecture, issues they faced, and how they iterated on their data model and use of Redis over time. It also discusses how Lyft uses Redis across their platform, including operations, monitoring, capacity planning, and their work contributing to open source projects like Envoy.
Hadoop Meetup Jan 2019 - Hadoop On AzureErik Krogen
ย
รรฑigo Goiri of Microsoft presents regarding the state of running Hadoop on Azure, Microsoft's cloud computing platform. He discusses some of the advanced features of Azure for cheaply running offline workloads, and what modifications have been made to Hadoop to take advantage of this functionality.
This is taken from the Apache Hadoop Contributors Meetup on January 30, hosted by LinkedIn in Mountain View.
Making Ceph awesome on Kubernetes with Rook - Bassam TabbaraCeph Community
ย
Rook makes running Ceph storage on Kubernetes easy by extending Kubernetes with custom controllers and types. It automates the deployment, configuration, scaling, upgrading and management of Ceph clusters running within Kubernetes. Rook defines the desired state of the storage cluster and uses an operator to reconcile the actual cluster state with the desired state. This allows Ceph to leverage the full power of Kubernetes' services, deployments and APIs for managing stateful applications at scale.
Ceph Client librbd Performance Analysis and Learnings - Mahati ChamarthyCeph Community
ย
This document summarizes an analysis of Ceph client (librbd) performance. It discusses RBD features, tracing and profiling tools used in testing, the test environment setup with a single OSD and Mon, and different workload configurations tested, including random read/write with different block sizes. Performance data was collected using eventtracing and analyzed to draw conclusions.
Messaging, interoperability and log aggregation - a new frameworkTomas Doran
ย
In this talk, I will talk about why log files are horrible, logging log lines, and more structured performance metrics from large scale production applications as well as building reliable, scaleable and flexible large scale software systems in multiple languages.
Why (almost) all log formats are horrible will be explained, and why JSON is a good solution for logging will be discussed, along with a number of message queuing, middleware and network transport technologies, including STOMP, AMQP and ZeroMQ.
The Message::Passing framework will be introduced, along with the logstash.net project which the perl code is interoperable with. These are pluggable frameworks in ruby/java/jruby and perl with pre-written sets of inputs, filters and outputs for many many different systems, message formats and transports.
They were initially designed to be aggregators and filters of data for logging. However they are flexible enough to be used as part of your messaging middleware, or even as a replacement for centralised message queuing systems.
You can have your cake and eat it too - an architecture which is flexible, extensible, scaleable and distributed. Build discrete, loosely coupled components which just pass messages to each other easily.
Integrate and interoperate with your existing code and code bases easily, consume from or publish to any existing message queue, logging or performance metrics system you have installed.
Simple examples using common input and output classes will be demonstrated using the framework, as will easily adding your own custom filters. A number of common messaging middleware patterns will be shown to be trivial to implement.
Some higher level use-cases will also be explored, demonstrating log indexing in ElasticSearch and how to build a responsive platform API using webhooks.
Interoperability is also an important goal for messaging middleware. The logstash.net project will be highlighted and we'll discuss crossing the single language barrier, allowing us to have full integration between java, ruby and perl components, and to easily write bindings into libraries we want to reuse in any of those languages.
This document discusses Scality's experiences building their first Node.js project. It summarizes that the project was building a TiVo-like cloud service for 25 million users, which required high parallelism and throughput of terabytes per second. It also discusses lessons learned around logging performance, optimizing the event loop and buffers, and useful Node.js tools.
Managing 50K+ Redis Databases Over 4 Public Clouds ... with a Tiny Devops TeamRedis Labs
ย
Redis is an in-memory database that provides fast performance for powering lightning fast apps. It supports many data structures like strings, hashes, lists, sets and sorted sets. Redis is efficient due to its support for many data structures and commands, as well as its complexity-aware design. Redis Labs provides fully-managed cloud services for Redis and Memcached, and helps customers with challenges around scalability, high availability, performance and monitoring for large-scale Redis deployments.
The document discusses how Redis is used at Facebook to solve two problems:
1) Serving user configurations in real-time by using a hierarchical master-slave replication architecture with global and regional masters and slaves to distribute reads and allow writes at different levels.
2) Processing stats data in real-time by using a split data processing model where daily bulk processing is done separately from real-time processing by tailers that insert stats into a sharded Redis cluster using HINCRBY.
How we got to 1 millisecond latency in 99% under repair, compaction, and flus...ScyllaDB
ย
Scylla is an open source reimplementation of Cassandra which performs up to 10X with drop in-replacement compatibility. At ScyllaDB, performance matters but even more importantly, stable performance under any circumstances.
A key factor for our consistent performance is our reliance on userspace schedulers. Scheduling in userspace allows the application, the database in our case to have better control on the different priorities each task has and to provide an SLA to selected operations. Scylla used to have an I/O scheduler and recently won a CPU scheduler.
At ScyllaDB, we make architectural decisions that provide not only low latencies but consistently low latencies at higher percentiles. This begins with our choice of language and key architectural decisions such as not using the Linux page-cache, and is fulfilled by autonomous database control, a set of algorithms, which guarantees that the system will adapt to changes in the workload. In the last year, we have made changes to Scylla that provide latencies that are consistent in every percentile. In this talk, Dor Laor will recap those changes and discuss what ScyllaDB is doing in the future.
Everything you wanted to know about RadosGW - Orit Wasserman, Matt BenjaminCeph Community
ย
This document provides an overview of object storage concepts and features of Ceph's RADOS Gateway (RGW). Key points include:
- RGW provides a RESTful API for object storage that is compatible with AWS S3 and OpenStack Swift.
- Objects stored in RGW can be large in size and immutable, with permissions set at the object level.
- Multipart uploads allow efficient transfer of large objects by splitting them into parts.
- Versioning and lifecycle policies allow automatic management of object versions and transitions.
- RGW can be used to provide NFS access to the object storage namespace.
Red Hat's storage roadmap aims to offer a unified, open software-defined storage portfolio for next generation workloads. Their portfolio includes Ceph and Gluster storage, which provide data services like object storage, block storage, and file sharing on standard hardware. Red Hat is working to improve management, performance, security, and integration of these products. They are also exploring new workloads like running MySQL on Ceph block storage and using Gluster Storage for hyperconverged Red Hat Enterprise Virtualization environments.
DataEngConf SF16 - BYOMQ: Why We [re]Built IronMQHakka Labs
ย
The document discusses reasons for reimplementing messaging queue functionality instead of using an existing solution like RabbitMQ or Kafka. It notes that reimplementing provides full understanding and control, allowing for quick fixes and easy addition of features without needing a daemon. However, it also acknowledges that reimplementing takes a long time to reach stability. The document then describes the specific requirements for the reimplemented messaging queue and provides a brief overview of its design and performance.
Scylla Summit 2018: Consensus in Eventually Consistent DatabasesScyllaDB
ย
Eventually consistent databases choose to remain available under failure, allowing for conflicting data to be stored in different replicas (later repaired by background processes). Weakening the consistency guarantees improves not only availability, but also performance, as the number of replicas involved in a given operation can be minimized. There are, however, use-cases that require the opposite trade-off. Indeed, Apache Cassandra and Scylla provide Lightweight Transactions (LWT), which allow single-key linearizable updates. The mechanism underlying LWT is asynchronous consensus. In this talk, we'll describe the characteristics and requirements of Scylla's consensus implementation, and how it enables strongly consistent updates. We will also cover how consensus can be applied to other aspects of the system, such as schema changes, node membership, and range movements, in order to improve their reliability and safety. We will thus show that an eventually consistent database can leverage consensus without compromising either availability or performance.
RedisConf17 - Redis in High Traffic Adtech StackRedis Labs
ย
Rahul Babbar presented on Times Internet's use of Redis in their high-traffic adtech stack. Some key points:
- Times Internet is India's largest digital company with 240+ million unique visitors per month.
- Their adtech stack powers ads for 150+ publishers and handles 9 billion ad impressions per month.
- Redis is used for caching, user profiles, analytics and more due to its low latency and ability to handle high volumes.
- Their Redis clusters consist of over 160 nodes across 4 clusters, serving over 2 million queries per second with 99% under 2ms response times.
What's new with enterprise Redis - Leena Joshi, Redis LabsRedis Labs
ย
Redis Labs manages over 160k+ HA databases, 10k clustered databases, without data loss in spite of one node failure a day and one data center outage per month. Using Enterprise
Redis(RLEC), Redis Labs delivers seamless zero downtime scaling, true high availability with persistence, cross-rack/zone/
datacenter replication and instant automatic failover. Learn how. Join this session for a deep dive into how enterprise Redis makes for no-hassle Redis deployments and the roadmap for new Redis capabilities. Discover new cost savings with Redis on Flash for cost-effective high performance operations and analytics
Experiences building a distributed shared log on RADOS - Noah WatkinsCeph Community
ย
This document summarizes Noah Watkins' presentation on building a distributed shared log using Ceph. The key points are:
1) Noah discusses how shared logs are challenging to scale due to the need to funnel all writes through a total ordering engine. This bottlenecks performance.
2) CORFU is introduced as a shared log design that decouples I/O from ordering by striping the log across flash devices and using a sequencer to assign positions.
3) Noah then explains how the components of CORFU can be mapped onto Ceph, using RADOS object classes, librados, and striping policies to implement the shared log without requiring custom hardware interfaces.
4) ZLog is presented
Linux Block Cache Practice on Ceph BlueStore - Junxin ZhangCeph Community
ย
This document discusses using Linux block caching with Ceph BlueStore. It explains that BlueStore can better utilize fast storage devices like SSDs compared to FileStore. It tested using Bcache and DM-writeboost to cache BlueStore data on HDDs using SSDs. Bcache performed better overall. Issues found were slow requests when caching and BlueStore used the same SSD, and inconsistency in SSD data management between BlueStore and the cache. Future work could have BlueStore control all raw disks and prioritize data saving to fast devices.
RADOS improvements and roadmap - Greg Farnum, Josh Durgin, Kefu ChaiCeph Community
ย
Cephalocon APAC 2018
March 22-23, 2018 - Beijing, China
Greg Farnum๏ผRed Hat RADOS Core Developer
Josh Durgin, Red Hat RADOS Lead
Kefu Chai, Red Hat Senior Software Engineer
Petabyte Scale Object Storage Service Using Ceph in A Private Cloud - Varada ...Ceph Community
ย
This document discusses Flipkart's use of Ceph object storage to provide a petabyte-scale object storage service. Key points:
- Flipkart runs two large data centers hosting over 20,000 servers and 60,000 VMs to power its e-commerce marketplace.
- It developed a highly scalable object storage service using Ceph to store over 1.5 billion objects totaling around 2PB of data. This service provides APIs compatible with AWS S3.
- The Ceph clusters are deployed across SSDs and HDDs to provide different performance and cost tiers for shared active, backup, and archival workloads with different SLAs around latency, throughput, and durability
Kafka Summit SF 2017 - Shopify Flash Sales with Apache Kafkaconfluent
ย
This document discusses how Shopify uses Apache Kafka in their systems architecture. It describes how Kafka provides reliable asynchronous messaging that allows Shopify to collect logs and events and feed them to their data lake. It outlines how Kafka provides operational decoupling and allows them to deploy application or Kafka changes independently. It then discusses two specific use cases: using Kafka as part of a logs/events pipeline and using it to enable active-active Elasticsearch replication across multiple data centers.
RedisConf17 - Lyft - Geospatial at Scale - Daniel HochmanRedis Labs
ย
This document discusses Lyft's migration from a monolithic architecture to using Redis to power a geospatial indexing system at scale. It describes Lyft's original architecture, issues they faced, and how they iterated on their data model and use of Redis over time. It also discusses how Lyft uses Redis across their platform, including operations, monitoring, capacity planning, and their work contributing to open source projects like Envoy.
Hadoop Meetup Jan 2019 - Hadoop On AzureErik Krogen
ย
รรฑigo Goiri of Microsoft presents regarding the state of running Hadoop on Azure, Microsoft's cloud computing platform. He discusses some of the advanced features of Azure for cheaply running offline workloads, and what modifications have been made to Hadoop to take advantage of this functionality.
This is taken from the Apache Hadoop Contributors Meetup on January 30, hosted by LinkedIn in Mountain View.
Making Ceph awesome on Kubernetes with Rook - Bassam TabbaraCeph Community
ย
Rook makes running Ceph storage on Kubernetes easy by extending Kubernetes with custom controllers and types. It automates the deployment, configuration, scaling, upgrading and management of Ceph clusters running within Kubernetes. Rook defines the desired state of the storage cluster and uses an operator to reconcile the actual cluster state with the desired state. This allows Ceph to leverage the full power of Kubernetes' services, deployments and APIs for managing stateful applications at scale.
Ceph Client librbd Performance Analysis and Learnings - Mahati ChamarthyCeph Community
ย
This document summarizes an analysis of Ceph client (librbd) performance. It discusses RBD features, tracing and profiling tools used in testing, the test environment setup with a single OSD and Mon, and different workload configurations tested, including random read/write with different block sizes. Performance data was collected using eventtracing and analyzed to draw conclusions.
Messaging, interoperability and log aggregation - a new frameworkTomas Doran
ย
In this talk, I will talk about why log files are horrible, logging log lines, and more structured performance metrics from large scale production applications as well as building reliable, scaleable and flexible large scale software systems in multiple languages.
Why (almost) all log formats are horrible will be explained, and why JSON is a good solution for logging will be discussed, along with a number of message queuing, middleware and network transport technologies, including STOMP, AMQP and ZeroMQ.
The Message::Passing framework will be introduced, along with the logstash.net project which the perl code is interoperable with. These are pluggable frameworks in ruby/java/jruby and perl with pre-written sets of inputs, filters and outputs for many many different systems, message formats and transports.
They were initially designed to be aggregators and filters of data for logging. However they are flexible enough to be used as part of your messaging middleware, or even as a replacement for centralised message queuing systems.
You can have your cake and eat it too - an architecture which is flexible, extensible, scaleable and distributed. Build discrete, loosely coupled components which just pass messages to each other easily.
Integrate and interoperate with your existing code and code bases easily, consume from or publish to any existing message queue, logging or performance metrics system you have installed.
Simple examples using common input and output classes will be demonstrated using the framework, as will easily adding your own custom filters. A number of common messaging middleware patterns will be shown to be trivial to implement.
Some higher level use-cases will also be explored, demonstrating log indexing in ElasticSearch and how to build a responsive platform API using webhooks.
Interoperability is also an important goal for messaging middleware. The logstash.net project will be highlighted and we'll discuss crossing the single language barrier, allowing us to have full integration between java, ruby and perl components, and to easily write bindings into libraries we want to reuse in any of those languages.
This document discusses Scality's experiences building their first Node.js project. It summarizes that the project was building a TiVo-like cloud service for 25 million users, which required high parallelism and throughput of terabytes per second. It also discusses lessons learned around logging performance, optimizing the event loop and buffers, and useful Node.js tools.
Slide deck for the Kubernetes Manchester meetup December 2018 talk. Jim introduces a little about moneysupermarket, the direction we're heading and historical problems we've had.
I (David) then walk through the technology choices we've made and how they fit together to form our Istio service mesh on an auto-scaling AWS EC2 kubernetes platform.
This document provides an overview and agenda for a two day Docker training course. Day one covers Docker introduction, installation, working with containers and images, building images with Dockerfiles, OpenStack integration, and Kubernetes introduction. Day two covers Docker cluster, Kubernetes in more depth, Docker networking, DockerHub, Docker use cases, and developing platforms with Docker. The document also includes sections on Docker basics, proposed cluster implementation strategies, and Kubernetes concepts and design principles.
Presto is an interactive SQL query engine for big data that was originally developed at Facebook in 2012 and open sourced in 2013. It is 10x faster than Hive for interactive queries on large datasets. Presto is highly extensible, supports pluggable backends, ANSI SQL, and complex queries. It uses an in-memory parallel processing architecture with pipelined task execution, data locality, caching, JIT compilation, and SQL optimizations to achieve high performance on large datasets.
Presentation of ActiveState Stackato to Copenhagen Ruby Brigade
More information available at: https://logiclab.jira.com/wiki/display/OPEN/Stackato
An Introduction to the Laravel Framework (AFUP Forum PHP 2014)daylerees
ย
This document is an introduction to the Laravel PHP framework presented by Dayle Rees. It discusses Laravel's history and growth from 2011 to present, how Rees became involved as an early user and contributor, and key features of Laravel like its dependency injection container, Eloquent ORM, queues, authentication, caching and more. It also promotes resources like Laravel Forge and Homestead for getting started with Laravel development.
Cassandra Day SV 2014: Spark, Shark, and Apache CassandraDataStax Academy
ย
Evan Chan from Ooyala presents on integrating Apache Spark and Apache Cassandra for interactive analytics. He discusses how Ooyala uses Cassandra for analytics and is becoming a major Spark user. The talk focuses on using Spark to generate dynamic queries over Cassandra data, as precomputing all possible aggregates is infeasible at Ooyala's scale. Chan describes Ooyala's architecture that uses Spark to generate materialized views from Cassandra for fast querying, and demonstrates running queries over a Spark/Cassandra dataset.
DrupalCampLA 2014 - Drupal backend performance and scalabilitycherryhillco
ย
This document discusses various techniques for optimizing Drupal backend performance and scalability. It covers diagnosing issues through tools like Apache Benchmark and Munin, optimizing hardware, web and database servers like using Nginx, Varnish, MySQL tuning, and alternative databases like MongoDB. It also discusses PHP optimizations like opcode caching and HHVM. The goal is to provide strategies to handle more traffic, improve page response times, and minimize downtime through infrastructure improvements and code optimizations.
Redis is an in-memory key-value data store that can be used for caching, sessions, queues, leaderboards, and more. It provides fast performance due to being memory-resident and supporting different data structures like strings, hashes, lists, sets, and sorted sets. Redis is useful for read-heavy and real-time applications but may not be suitable if data does not fit in memory or for relational data needs. The presentation discusses using Redis with PHP and Symfony, data sharding strategies, and war stories from a social game with 7.5M daily users.
Building a reliable pipeline of data ingress, batch computation, and data egress with Hadoop can be a major challenge. Most folks start out with cron to manage workflows, but soon discover that doesn't scale past a handful of jobs. There are a number of open-source workflow engines with support for Hadoop, including Azkaban (from LinkedIn), Luigi (from Spotify), and Apache Oozie. Having deployed all three of these systems in production, Joe will talk about what features and qualities are important for a workflow system.
Node.js architecture is all about managing the incoming requests and providing an appropriate response to each of those requests. This article dives deeper into the working of Node.js and the workflow of the whole server environment.
"WTF is Twisted? (or; owl amongst the ponies)" is a talk that introduces the Twisted asynchronous programming framework, how it works, and what uses it.
Reuven Lerner's first talk from Open Ruby Day, at Hi-Tech College in Herzliya, Israel, on June 27th 2010. An overview of what makes Rails a powerful framework for Web development -- what attracted Reuven to it, what are the components that most speak to him, and why others should consider Rails for their Web applications.
Server-side JavaScript (SSJS) is gaining popularity due to factors like the rise of NoSQL databases, asynchronous programming, and JavaScript's ubiquity. SSJS environments like Node.js, CommonJS, and AppEngineJS allow developers to use JavaScript beyond the browser by running it on the server. Google App Engine also provides a platform for hosting SSJS applications and automatically scaling them.
Modern software architectures - PHP UK Conference 2015Ricard Clau
ย
The web has changed. Users demand responsive, real-time interactive applications and companies need to store and analyze tons of data. Some years ago, monolithic code bases with a basic LAMP stack, some caching and perhaps a search engine were enough. These days everybody is talking about micro-services architectures, SOA, Erlang, Golang, message passing, queue systems and many more. PHP seems to not be cool anymore but... is this true? Should we all forget everything we know and just learn these new technologies? Do we really need all these things?
This webinar covered Neo4j 1.9.1, 2.0 features and hands-on demonstrations. Key highlights included:
- Neo4j 1.9.1 improvements to high availability, operations and Cypher performance.
- Neo4j 2.0 introduces node labels for improved querying and indexing, transactional Cypher via REST, and a focus on using Cypher everywhere.
- Hands-on sessions demonstrated migrating data to use labels, indexing, transactional HTTP endpoint, and other new 2.0 features.
Redis Day Bangalore 2020 - Session state caching with redisRedis Labs
ย
This document discusses using Redis caching to improve performance for the DBS Paylah mobile wallet application. Paylah aims to significantly increase its user base which will increase load on its backend systems. Caching application data and session state in Redis can reduce latency, improve responsiveness for users, and reduce costs by lowering load on legacy backend databases and mainframes. The document outlines some key Paylah use cases where caching transaction histories and account details in Redis would accelerate retrieval and improve the mobile experience by avoiding the need to access slower backend systems on each request.
Protecting Your API with Redis by Jane Paek - Redis Day Seattle 2020Redis Labs
ย
The document discusses rate limiting and metering using Redis. It begins by introducing rate limiting and metering and why Redis is well-suited for these tasks. It then covers different Redis data structures that can be used, such as lists, hashes, sorted sets and strings. Common Redis commands for counting, setting keys and checking time to live are also presented. Different rate limiting design patterns and anti-patterns are described, including fixed window, sliding window and token bucket approaches. Finally, resources for further information are provided.
The Happy Marriage of Redis and Protobuf by Scott Haines of Twilio - Redis Da...Redis Labs
ย
The document summarizes a presentation about using Protocol Buffers and Redis together. It discusses how Protocol Buffers provide strict data types, versioning, and serialization/deserialization benefits. It then outlines Redis key patterns using namespaces, versions, data categories and identifiers. Examples are provided to show how Protocol Buffers messages can be stored in Redis using these key patterns, including storing connections data in sets, sorted sets and individual messages. Benefits discussed include structure, readability, testing and abstraction.
SQL, Redis and Kubernetes by Paul Stanton of Windocks - Redis Day Seattle 2020Redis Labs
ย
The document discusses common use cases for combining SQL, Redis, and Kubernetes including caching, session management, rate limiting, and data ingestion. It outlines how Kubernetes can be used for scaling microservices while Redis is used for data service scaling. The presentation proposes combining Redis, SQL Server, and Kubernetes with a proxy service, and describes using Redis for caching, session storage, and rate limiting of SQL data. It also discusses running Redis and front-end apps on Kubernetes and deploying SQL as a Kubernetes service through a proxy.
Rust and Redis - Solving Problems for Kubernetes by Ravi Jagannathan of VMwar...Redis Labs
ย
This document discusses using Rust and Redis to build cloud native platforms. It first provides context about devops and the need to do more with less. It then discusses how platforms are becoming more distributed and Kubernetes upends distribution paradigms. The document dives into how Rust addresses issues like concurrency and systems programming. It also discusses how Redis can be used for caching, queues, streams and more. Finally, it mentions that Rust and Redis will be demonstrated.
Redis for Data Science and Engineering by Dmitry Polyakovsky of OracleRedis Labs
ย
This document contains a presentation about using Redis for data science and engineering. It introduces the presenter and provides an agenda that covers using Redis for data science and data engineering. The presentation notes that Redis can be used as both a data store and job queue, has flexible data structures and is fast, though it uses RAM and cannot query by value. It also lists Python Pandas and includes a demo and links for further information.
Practical Use Cases for ACLs in Redis 6 by Jamie Scott - Redis Day Seattle 2020Redis Labs
ย
Jamie Scott from RedisLabs presented on practical use cases for access control lists (ACLs) in Redis 6. The presentation covered new security features in Redis 6 including encryption in transit, key space and command restrictions, and multiple access control list users. It demonstrated how ACLs allow users to define access based on key labels and restrictions. ACLs can facilitate discretionary and mandatory access controls. The presentation showed examples of using ACLs to restrict user access by key labels and commands to enhance operational security.
Moving Beyond Cache by Yiftach Shoolman Redis Labs - Redis Day Seattle 2020Redis Labs
ย
This document summarizes a presentation about Redis version 6 and beyond. Some key points include:
- Redis version 6 includes new features like ACL for security, client-side caching, diskless replication, and multi-threaded I/O.
- Redis is positioned as both a cache and a database due to its speed, data structures, and ability to handle complex data models through modules.
- Redis Enterprise provides additional capabilities like durability, high availability, geo-distribution, security and multi-tenancy.
- Modern data models in Redis modules include Streams, RediSearch, RedisGraph, RedisTimeSeries, RedisAI, RedisJSON and RedisBloom.
- RedisInsight is
Leveraging Redis for System Monitoring by Adam McCormick of SBG - Redis Day S...Redis Labs
ย
The document discusses how Sinclair Broadcast Group leverages Redis for system monitoring of its content delivery network. It operates 193 news stations with 10,000 active pages daily and millions in archive. New stories are posted every 15 seconds and must be visible across its 1,000+ targets within 1 minute. Redis is used to track performance across the multi-level CDN and ensure service level agreements are met with real-time resolution and alerting. It provides a black box view of the audience experience and can scale monitoring to all relevant pages within 30 seconds. Redis acts as a distributed data store to parallelize the monitoring task across the large scale of the network.
JSON in Redis - When to use RedisJSON by Jay Won of Coupang - Redis Day Seatt...Redis Labs
ย
The document summarizes a presentation about when to use the RedisJSON data type. It discusses how Coupang uses Redis extensively for their ad platform. It then compares the performance and memory usage of storing JSON data as strings, hashes, or using the RedisJSON data type. Benchmark results show RedisJSON can provide better performance for retrieving and updating JSON fields compared to strings and hashes, though it uses more memory. The document recommends using RedisJSON for smaller JSON payloads after benchmarking and memory monitoring.
Highly Available Persistent Session Management Service by Mohamed Elmergawi o...Redis Labs
ย
The document discusses the challenges of building a highly available persistent session management service. It describes Zulily's legacy architecture which lacked high availability and required manual intervention. A new architecture is proposed using Redis for persistent storage, Dynomite for real-time replication across data centers, and a connection pooling proxy to improve efficiency and distribute load. The architecture provides high availability through replication, reduces overhead through connection pooling, and handles failures through consistent hashing and health checks. It was tested through simulations and showed a failure rate of only 0.42% during outages.
Anatomy of a Redis Command by Madelyn Olson of Amazon Web Services - Redis Da...Redis Labs
ย
The document describes the process that a Redis command follows from the client side to the server side. On the client side, the command is sent over the network to the Redis server. On the server side, the command is read from the kernel buffers, validated, executed by calling the relevant command handler, and the response is written back to the client over the network. The core functions involved on the server side are ReadQueryFromClient(), ProcessInputBuffer(), ProcessCommand(), Call(), and handleClientsWithPendingWrites(). Redis 6.0 introduced I/O threads to handle reads and writes in parallel for improved performance while still maintaining Redis' single-threaded processing model.
Building a Multi-dimensional Analytics Engine with RedisGraph by Matthew Goos...Redis Labs
ย
This document discusses MDmetrix, a healthcare data intelligence company that uses RedisGraph to provide flexible analysis of hospital data. RedisGraph is a graph database that represents data as nodes and relationships and uses an adjacency matrix and linear algebra to query the graph. MDmetrix models its healthcare data as a property graph in RedisGraph to allow for complex queries across different data dimensions like patients, facilities, procedures and drugs. RedisGraph allows MDmetrix to query the data more easily than traditional OLAP cubes or relational databases due to the semi-structured and flexible nature of the graph model.
RediSearch 1.6 by Pieter Cailliau - Redis Day Bangalore 2020Redis Labs
ย
RediSearch 1.6 includes a new low-level API that allows other Redis modules to embed RediSearch indexing capabilities. It also introduces index aliasing and several performance improvements such as forked thread garbage collection. Based on benchmarks, RediSearch 1.6 shows 48-73% better performance than version 1.4, particularly during high update rates where it maintains more stable read latencies.
RedisGraph 2.0 by Pieter Cailliau - Redis Day Bangalore 2020Redis Labs
ย
RedisGraph 2.0 provides significant improvements including:
- Full text search support through embedded RediSearch 1.6 enabling graph-aided search.
- Support for returning full graph responses to enable better visualization.
- Broad support for Cypher including triadic closure and new graph-aided search capabilities.
- Performance improvements of up to 3.7x faster operations per second and 3.9x faster query times compared to RedisGraph v1.2.
- Support for benchmarking including the LDBC benchmark.
RedisTimeSeries 1.2 by Pieter Cailliau - Redis Day Bangalore 2020Redis Labs
ย
RedisTimeSeries is a time-series database that provides compression to reduce memory usage by up to 98% and improve performance. The RedisTimeSeries 1.2 release includes compression algorithms based on a Facebook paper that provide stable ingestion times independent of the number of data points. It also includes a reviewed API with performance improvements and clearer functionality. Performance testing showed ingestion throughput improved by 2-3% and query performance increased from 15-70% with the new release compared to the previous version.
RedisAI 0.9 by Sherin Thomas of Tensorwerk - Redis Day Bangalore 2020Redis Labs
ย
This document summarizes RedisAI 0.9 and its capabilities for model deployment and benchmarking. It introduces RedisAI's new tensor data type and ability to deploy models to CPU and GPU. It then discusses AIBench, a tool developed to benchmark AI serving solutions like RedisAI, TensorFlow Serving, and REST APIs. The benchmarks show RedisAI providing 5.5x and 2.5x more inferences than REST APIs and TensorFlow Serving respectively, due to its data locality. The document concludes by mentioning RedisAI's integration with MLFlow for model deployment with a single command.
Rate-Limiting 30 Million requests by Vijay Lakshminarayanan and Girish Koundi...Redis Labs
ย
The document discusses how Freshworks uses Redis Labs to rate limit 30 million API requests per day through their API gateway called Fluffy. Fluffy stores rate limit policies and maintains counters to track requests. Redis Labs allows Fluffy to easily scale to handle the high volume of requests by providing a fast, in-memory data store for managing rate limiting counters. The system was able to successfully rate limit 30 million requests per day with Redis Labs.
Solving Complex Scaling Problems by Prashant Kumar and Abhishek Jain of Myntr...Redis Labs
ย
Redis was used by Myntra to solve several complex scaling problems. It was used to build a scalable user segment service to support high read throughput of up to 5 million requests per minute with low latency. Redis allowed the service to scale beyond a single instance and included features like automatic backups and memory management. Redis also helped build a scalable mobile verification platform to reliably handle 100,000 requests per minute and scale to support higher future volumes. It was used as both a transient store and persistent backend. Finally, Redis locks helped build a scalable A/B testing platform by allowing experiments to be created and updated in an orderly concurrent fashion.
Semantic Cultivators : The Critical Future Role to Enable AIartmondano
ย
By 2026, AI agents will consume 10x more enterprise data than humans, but with none of the contextual understanding that prevents catastrophic misinterpretations.
What is Model Context Protocol(MCP) - The new technology for communication bw...Vishnu Singh Chundawat
ย
The MCP (Model Context Protocol) is a framework designed to manage context and interaction within complex systems. This SlideShare presentation will provide a detailed overview of the MCP Model, its applications, and how it plays a crucial role in improving communication and decision-making in distributed systems. We will explore the key concepts behind the protocol, including the importance of context, data management, and how this model enhances system adaptability and responsiveness. Ideal for software developers, system architects, and IT professionals, this presentation will offer valuable insights into how the MCP Model can streamline workflows, improve efficiency, and create more intuitive systems for a wide range of use cases.
Rock, Paper, Scissors: An Apex Map Learning JourneyLynda Kane
ย
Slide Deck from Presentations to WITDevs (April 2021) and Cleveland Developer Group (6/28/2023) on using Rock, Paper, Scissors to learn the Map construct in Salesforce Apex development.
Big Data Analytics Quick Research Guide by Arthur MorganArthur Morgan
ย
This is a Quick Research Guide (QRG).
QRGs include the following:
- A brief, high-level overview of the QRG topic.
- A milestone timeline for the QRG topic.
- Links to various free online resource materials to provide a deeper dive into the QRG topic.
- Conclusion and a recommendation for at least two books available in the SJPL system on the QRG topic.
QRGs planned for the series:
- Artificial Intelligence QRG
- Quantum Computing QRG
- Big Data Analytics QRG
- Spacecraft Guidance, Navigation & Control QRG (coming 2026)
- UK Home Computing & The Birth of ARM QRG (coming 2027)
Any questions or comments?
- Please contact Arthur Morgan at [email protected].
100% human made.
This is the keynote of the Into the Box conference, highlighting the release of the BoxLang JVM language, its key enhancements, and its vision for the future.
Procurement Insights Cost To Value Guide.pptxJon Hansen
ย
Procurement Insights integrated Historic Procurement Industry Archives, serves as a powerful complement โ not a competitor โ to other procurement industry firms. It fills critical gaps in depth, agility, and contextual insight that most traditional analyst and association models overlook.
Learn more about this value- driven proprietary service offering here.
AI EngineHost Review: Revolutionary USA Datacenter-Based Hosting with NVIDIA ...SOFTTECHHUB
ย
I started my online journey with several hosting services before stumbling upon Ai EngineHost. At first, the idea of paying one fee and getting lifetime access seemed too good to pass up. The platform is built on reliable US-based servers, ensuring your projects run at high speeds and remain safe. Let me take you step by step through its benefits and features as I explain why this hosting solution is a perfect fit for digital entrepreneurs.
How Can I use the AI Hype in my Business Context?Daniel Lehner
ย
๐๐จ ๐ผ๐ ๐๐ช๐จ๐ฉ ๐๐ฎ๐ฅ๐? ๐๐ง ๐๐จ ๐๐ฉ ๐ฉ๐๐ ๐๐๐ข๐ ๐๐๐๐ฃ๐๐๐ง ๐ฎ๐ค๐ช๐ง ๐๐ช๐จ๐๐ฃ๐๐จ๐จ ๐ฃ๐๐๐๐จ?
Everyoneโs talking about AI but is anyone really using it to create real value?
Most companies want to leverage AI. Few know ๐ต๐ผ๐.
โ What exactly should you ask to find real AI opportunities?
โ Which AI techniques actually fit your business?
โ Is your data even ready for AI?
If youโre not sure, youโre not alone. This is a condensed version of the slides I presented at a Linkedin webinar for Tecnovy on 28.04.2025.
Buckeye Dreamin 2024: Assessing and Resolving Technical DebtLynda Kane
ย
Slide Deck from Buckeye Dreamin' 2024 presentation Assessing and Resolving Technical Debt. Focused on identifying technical debt in Salesforce and working towards resolving it.
DevOpsDays Atlanta 2025 - Building 10x Development Organizations.pptxJustin Reock
ย
Building 10x Organizations with Modern Productivity Metrics
10x developers may be a myth, but 10x organizations are very real, as proven by the influential study performed in the 1980s, โThe Coding War Games.โ
Right now, here in early 2025, we seem to be experiencing YAPP (Yet Another Productivity Philosophy), and that philosophy is converging on developer experience. It seems that with every new method we invent for the delivery of products, whether physical or virtual, we reinvent productivity philosophies to go alongside them.
But which of these approaches actually work? DORA? SPACE? DevEx? What should we invest in and create urgency behind today, so that we donโt find ourselves having the same discussion again in a decade?
Spark is a powerhouse for large datasets, but when it comes to smaller data workloads, its overhead can sometimes slow things down. What if you could achieve high performance and efficiency without the need for Spark?
At S&P Global Commodity Insights, having a complete view of global energy and commodities markets enables customers to make data-driven decisions with confidence and create long-term, sustainable value. ๐
Explore delta-rs + CDC and how these open-source innovations power lightweight, high-performance data applications beyond Spark! ๐
Leading AI Innovation As A Product Manager - Michael JidaelMichael Jidael
ย
Unlike traditional product management, AI product leadership requires new mental models, collaborative approaches, and new measurement frameworks. This presentation breaks down how Product Managers can successfully lead AI Innovation in today's rapidly evolving technology landscape. Drawing from practical experience and industry best practices, I shared frameworks, approaches, and mindset shifts essential for product leaders navigating the unique challenges of AI product development.
In this deck, you'll discover:
- What AI leadership means for product managers
- The fundamental paradigm shift required for AI product development.
- A framework for identifying high-value AI opportunities for your products.
- How to transition from user stories to AI learning loops and hypothesis-driven development.
- The essential AI product management framework for defining, developing, and deploying intelligence.
- Technical and business metrics that matter in AI product development.
- Strategies for effective collaboration with data science and engineering teams.
- Framework for handling AI's probabilistic nature and setting stakeholder expectations.
- A real-world case study demonstrating these principles in action.
- Practical next steps to begin your AI product leadership journey.
This presentation is essential for Product Managers, aspiring PMs, product leaders, innovators, and anyone interested in understanding how to successfully build and manage AI-powered products from idea to impact. The key takeaway is that leading AI products is about creating capabilities (intelligence) that continuously improve and deliver increasing value over time.
19. Lua new replication modes
โข Now you can write script with side effects, random
commands, โฆ
โข Selective replication of certain commands to
slaves.
20. Expires consistency
โข Slave and Master now agree much better about
what keys already expired.
โข Expires are still up to the master, but the slave, for
read operations, can show a sane behavior.
21. RDB improvements
โข RDB has now auxiliary informations inside.
โข Itโs faster to load since it resizes the hash table
before loading the keys.
โข Soโฆ no rehashing needed (is costly).
22. Geo commands
โข 49 commits, 899 insertions, 1429 deletions.
โข API simpli๏ฌed, JSON API removed entirely.
โข An useless indirection layer removed. Geo indexing
API taken from Ardb simpli๏ฌed & improved.
โข Radius queries ๏ฌxed to really report items in range.
โข Much faster (may be yet a lot fasterโฆ work needed)
23. Quicklists
โข Idea initially conceived by Pieter & me, never
implemented.
โข Later we saw Twitter implementing it, but just the
minimum they needed.
โข Finally Matt Stancliff provided a good
implementation.
โข Sun He and I re๏ฌned the implementation and
improved the on-disk serialization.
25. Improved ZADD
โข NX, XX, CH, INCR.
โข Better capabilities via options.
โข Newcomers can ignore them.
โข Complexity is disclosed incrementally.
โข Commands list does not becomes huge.
โข What about Markov Chains?
26. SPOP with count
โข Ability to pop multiple items from set in a single
call.
โข Small change, but required some reworking of the
replication code.
31. Lazyfree basics
โข It means: O(1) DEL synchronous work.
โข Plus: O(N) DEL asynchronous work.
โข The result: DEL is no longer blocking.
โข API: UNLINK and ASYNC option for FLUSHDB
and FLUSHALL.
33. More stuff
โข Better SDS library.
โข More useful and polished crash reports.
โข CLIENT REPLY to suppress replies for bulk inserts.
โข DEBUG RESTART and CRASH-AND-RECOVERY.
โข HSTRLEN command.
โข Too many to show them all.
35. Threaded I/O
โข Spawn N threads at startup.
โข Each thread does multiplexing like now.
โข Each client is pinned to a thread when connecting.
โข We can ๏ฌnally scale write, read, multiplexing.
โข Global Lock. Similar to memcached.
36. Threaded slow ops
โข Client is executing SUNION foo bar.
โข Block it (like in blocked.c).
โข Lock the key in a lock hash table: no mutex per
key! Single mutex for the lock hash table.
โข Perform operation, including sending the output
in the client output buffer, in a different thread.
โข Unblock the client.
37. N-dimensional range
queries
โข Already implemented as a Ruby library.
โข Uses bit to bit interpolation.
โข We want to add it as a native API.
โข NADD myindex 3 1.4 200.4 4321 element
โข NRANGE myindex [0 1] [10 20] (30 40]
38. Bloom ๏ฌlters
โข API is the thing that matters.
โข I would love no parameters (like HyperLogLogs).
โข Auto scaling, without false negatives.
โข Alternatives to bloom ๏ฌlters with same semantics
look promising compared to bloom ๏ฌlters.