Redis : Play buzz uses Redis

Mar 30, 20160 likes378 views

Chen Salomon introduces himself as a data architect at Playbuzz, a 3-year-old company with 100 employees and 30 in R&D. Playbuzz creates engaging content formats for over 7,000 publishers. To scale efficiently, Playbuzz uses Redis for distributed caching, ETL session management, IP location lookups, and more. Their most recent Redis project converts IP addresses to integers for fast location lookups from cached hashmaps.

A little bit about myself
Chen Salomon, Data architect– c@Playbuzz.com
First developer @ Playbuzz
The guy that still wakes when shit hits the fan…
Currently leading (data) solutions

Playbuzz
3 years old
4 locations (Israel, NY, London, Hamburg)
100 employees
30 in RnD (we are still small…)

Seconds on a page
60%78%15
Source: Wall Street Journal, 2014
Changes in internet content
of internet users don’t
read entire articles
Web publishers
get
of traffic through
referrals from
Facebook on
average
User stays only

Playbuzz is the king of Engaging Content™
7,000+
publishers can’t
be wrong

Formats
Gallery QuizTriviaPersonality QuizListRevealRanked List
Countdown List
Poll
Open List
Photo Caption
Video

• Keep it simple and super slim – We always try to keep our
production with as few servers as possible
• Be lazy – We pay a lot on insert and barely work on reads
• Know your traffic
• Enabling the business to scale at minimal cost
How we do it?

• Object caching
• Production usage
• Keeping items in cache (lazy initialization) – objects (strings)
• List management as sorted sets
• Permissions are hashmaps
Where you can find Redis in our system?

• Distributed HTML Output caching
• ETL session management and enrichments
• Session parameters are objects
• IP to location (in a few seconds)
Where you can find Redis in our system?

Most recent Redis project
IP to location
• Location objects are hash-maps
• Convert IP to integers
• e.g. IP 10.0.0.138 -> 10 *(256^3) + 0 *(256^2) + 0 *(256) + 138
*(256^0) =167772298
• ZRANGEBYSCORE to find the range (position) – returns the first item
that larger than the IP.
• HGETALL for the returned position
• Challenges
• IP V6 (same idea –different multiplier)
• Empty subnets – You can’t use only lower/upper bounds

This document summarizes a presentation on using Riak to solve hard distributed systems problems. It discusses Riak's key features and tradeoffs in consistency and data model. It then provides examples of how companies like Comcast, Yammer, and Github use Riak for high availability and low latency applications like user metadata storage, notifications, and as a document store. The presentation emphasizes that choosing the right database requires understanding your problem and tradeoffs in consistency, availability and data model. It also highlights how Riak and Erlang help enable high availability, low latency and operational simplicity through features like Bitcask storage, fault tolerance and lightweight threading.

Recommendations with Neo4j (FOSDEM 2015)Michal Bachman

This document discusses recommendations engines that use graph databases like Neo4j. It introduces GraphAware, an open-source recommendation engine plugin for Neo4j. The document outlines the business and technical challenges of building recommendation engines, and how GraphAware addresses these challenges through its flexible, high-performance architecture and APIs. It provides an example of building a simple friend recommendation engine using GraphAware.

Advanced Neo4j Use Cases with the GraphAware FrameworkMichal Bachman

The document discusses GraphAware Framework, which makes it easy to build, test, and deploy custom APIs, transaction-driven behavior, and asynchronous computation functionality for Neo4j. It provides examples like representing time series data, tracking graph changes, assigning UUIDs, and running algorithms. GraphAware Framework is open source and supports building both generic and domain-specific Neo4j extensions.

Modelling Data in Neo4j (plus a few tips)Michal Bachman

Redis use casesBottom Wang

Best Buy Web 2.0Lee Aase

Introduction to Redis Data Structures: Sorted SetsScaleGrid.io

Redis and its many use casesChristian Joudrey

Redis is an open source, advanced key-value store that can be used for caching, sessions, publishing/subscribing, job queues, and more. It supports atomic operations, ordered lists, sets, hashes and sorted sets. Data can be saved to disk for persistence. Redis has client libraries for many languages like Node.js and can be accessed via a command line interface. Popular use cases include caching application data to improve performance, storing sessions to allow user authentication across requests, and processing jobs asynchronously in the background via pub/sub and queues.

Introduction to some top Redis use casesJosiah Carlson

This document introduces top Redis use cases. It begins by introducing the speaker, Dr. Josiah Carlson, and providing a brief overview of what Redis is - an in-memory non-relational key-value store with optional persistence and replication. The top five use cases discussed are distributed locking, analytics, storing web cookies, queues/message passing, and anything previously done with Memcached. The document concludes by noting there are many additional use cases for Redis like caching, rankings, and as a general data store.

The BestBuy.com Cloud Architecturejoelcrabb

Using Redis at FacebookRedis Labs

The document discusses how Redis is used at Facebook to solve two problems: 1) Serving user configurations in real-time by using a hierarchical master-slave replication architecture with global and regional masters and slaves to distribute reads and allow writes at different levels. 2) Processing stats data in real-time by using a split data processing model where daily bulk processing is done separately from real-time processing by tailers that insert stats into a sharded Redis cluster using HINCRBY.

Bootstrapping Recommendations with Neo4jMax De Marzi

This document provides an overview of using Neo4j and graph databases for bootstrapping recommendations in real-time. It discusses common recommendation approaches like popularity, content-based, collaborative filtering and hybrid recommendations. It also covers challenges of real-time recommendations like processing relationships and accommodating new data continuously. Additionally, it demonstrates sample Cypher queries for calculating similarity between users and providing movie recommendations based on a user's nearest neighbors.

Neo4j GraphTalks - Einführung in GraphdatenbankenNeo4j

Redis Use Patterns (DevconTLV June 2014)Itamar Haber

Neo4j the Anti Crime DatabaseNeo4j

This document introduces Neo4j, the world's leading graph database. It discusses Neo4j's product and company details, how graph databases are different than other databases by focusing on relationships between connected data. Common use cases for Neo4j are also summarized, such as recommendations, master data management, network operations, identity and access management, and fraud detection. The document provides examples of how customers use Neo4j and discusses patterns of fraud that Neo4j can help detect.

Neo4j GraphTalks Panama PapersNeo4j

Intro to Neo4j and Graph DatabasesNeo4j

Everything you always wanted to know about Redis but were afraid to askCarlos Abalde

Neo4j - 5 cool graph examplesPeter Neubauer

This document discusses graph databases and provides examples of how the Neo4j graph database can be used. It shows how Neo4j supports social, spatial, financial and other types of connected data. It also summarizes Neo4j's REST API, support for object-oriented programming, routing algorithms, multiple indexes, recommendation systems, and other use cases. The document advocates for graph databases for any problem involving multiple relationships and connections between entities.

Graph database Use CasesMax De Marzi

This document provides an overview of graph databases and their use cases. It begins with definitions of graphs and graph databases. It then gives examples of how graph databases can be used for social networking, network management, and other domains where data is interconnected. It provides Cypher examples for creating and querying graph patterns in a social networking and IT network management scenario. Finally, it discusses the graph database ecosystem and how graphs can be deployed for both online transaction processing and batch processing use cases.

Redis Day Bangalore 2020 - Session state caching with redisRedis Labs

This document discusses using Redis caching to improve performance for the DBS Paylah mobile wallet application. Paylah aims to significantly increase its user base which will increase load on its backend systems. Caching application data and session state in Redis can reduce latency, improve responsiveness for users, and reduce costs by lowering load on legacy backend databases and mainframes. The document outlines some key Paylah use cases where caching transaction histories and account details in Redis would accelerate retrieval and improve the mobile experience by avoiding the need to access slower backend systems on each request.

Protecting Your API with Redis by Jane Paek - Redis Day Seattle 2020Redis Labs

The document discusses rate limiting and metering using Redis. It begins by introducing rate limiting and metering and why Redis is well-suited for these tasks. It then covers different Redis data structures that can be used, such as lists, hashes, sorted sets and strings. Common Redis commands for counting, setting keys and checking time to live are also presented. Different rate limiting design patterns and anti-patterns are described, including fixed window, sliding window and token bucket approaches. Finally, resources for further information are provided.

The Happy Marriage of Redis and Protobuf by Scott Haines of Twilio - Redis Da...Redis Labs

The document summarizes a presentation about using Protocol Buffers and Redis together. It discusses how Protocol Buffers provide strict data types, versioning, and serialization/deserialization benefits. It then outlines Redis key patterns using namespaces, versions, data categories and identifiers. Examples are provided to show how Protocol Buffers messages can be stored in Redis using these key patterns, including storing connections data in sets, sorted sets and individual messages. Benefits discussed include structure, readability, testing and abstraction.

SQL, Redis and Kubernetes by Paul Stanton of Windocks - Redis Day Seattle 2020Redis Labs

The document discusses common use cases for combining SQL, Redis, and Kubernetes including caching, session management, rate limiting, and data ingestion. It outlines how Kubernetes can be used for scaling microservices while Redis is used for data service scaling. The presentation proposes combining Redis, SQL Server, and Kubernetes with a proxy service, and describes using Redis for caching, session storage, and rate limiting of SQL data. It also discusses running Redis and front-end apps on Kubernetes and deploying SQL as a Kubernetes service through a proxy.

Rust and Redis - Solving Problems for Kubernetes by Ravi Jagannathan of VMwar...Redis Labs

This document discusses using Rust and Redis to build cloud native platforms. It first provides context about devops and the need to do more with less. It then discusses how platforms are becoming more distributed and Kubernetes upends distribution paradigms. The document dives into how Rust addresses issues like concurrency and systems programming. It also discusses how Redis can be used for caching, queues, streams and more. Finally, it mentions that Rust and Redis will be demonstrated.

Redis for Data Science and Engineering by Dmitry Polyakovsky of OracleRedis Labs

This document contains a presentation about using Redis for data science and engineering. It introduces the presenter and provides an agenda that covers using Redis for data science and data engineering. The presentation notes that Redis can be used as both a data store and job queue, has flexible data structures and is fast, though it uses RAM and cannot query by value. It also lists Python Pandas and includes a demo and links for further information.

Practical Use Cases for ACLs in Redis 6 by Jamie Scott - Redis Day Seattle 2020Redis Labs

Jamie Scott from RedisLabs presented on practical use cases for access control lists (ACLs) in Redis 6. The presentation covered new security features in Redis 6 including encryption in transit, key space and command restrictions, and multiple access control list users. It demonstrated how ACLs allow users to define access based on key labels and restrictions. ACLs can facilitate discretionary and mandatory access controls. The presentation showed examples of using ACLs to restrict user access by key labels and commands to enhance operational security.

More Related Content

Viewers also liked (15)

Best Buy Web 2.0Lee Aase

Introduction to Redis Data Structures: Sorted SetsScaleGrid.io

Redis and its many use casesChristian Joudrey

Introduction to some top Redis use casesJosiah Carlson

The BestBuy.com Cloud Architecturejoelcrabb

Using Redis at FacebookRedis Labs

Bootstrapping Recommendations with Neo4jMax De Marzi

Neo4j GraphTalks - Einführung in GraphdatenbankenNeo4j

Redis Use Patterns (DevconTLV June 2014)Itamar Haber

Neo4j the Anti Crime DatabaseNeo4j

Neo4j GraphTalks Panama PapersNeo4j

Intro to Neo4j and Graph DatabasesNeo4j

Everything you always wanted to know about Redis but were afraid to askCarlos Abalde

Neo4j - 5 cool graph examplesPeter Neubauer

Graph database Use CasesMax De Marzi

Best Buy Web 2.0Lee Aase

Introduction to Redis Data Structures: Sorted SetsScaleGrid.io

Redis and its many use casesChristian Joudrey

Introduction to some top Redis use casesJosiah Carlson

The BestBuy.com Cloud Architecturejoelcrabb

Using Redis at FacebookRedis Labs

Bootstrapping Recommendations with Neo4jMax De Marzi

Neo4j GraphTalks - Einführung in GraphdatenbankenNeo4j

Redis Use Patterns (DevconTLV June 2014)Itamar Haber

Neo4j the Anti Crime DatabaseNeo4j

Neo4j GraphTalks Panama PapersNeo4j

Intro to Neo4j and Graph DatabasesNeo4j

Everything you always wanted to know about Redis but were afraid to askCarlos Abalde

Neo4j - 5 cool graph examplesPeter Neubauer

Graph database Use CasesMax De Marzi

More from Redis Labs (20)

Redis Day Bangalore 2020 - Session state caching with redisRedis Labs

Protecting Your API with Redis by Jane Paek - Redis Day Seattle 2020Redis Labs

The Happy Marriage of Redis and Protobuf by Scott Haines of Twilio - Redis Da...Redis Labs

SQL, Redis and Kubernetes by Paul Stanton of Windocks - Redis Day Seattle 2020Redis Labs

Rust and Redis - Solving Problems for Kubernetes by Ravi Jagannathan of VMwar...Redis Labs

Redis for Data Science and Engineering by Dmitry Polyakovsky of OracleRedis Labs

Practical Use Cases for ACLs in Redis 6 by Jamie Scott - Redis Day Seattle 2020Redis Labs

Moving Beyond Cache by Yiftach Shoolman Redis Labs - Redis Day Seattle 2020Redis Labs

This document summarizes a presentation about Redis version 6 and beyond. Some key points include: - Redis version 6 includes new features like ACL for security, client-side caching, diskless replication, and multi-threaded I/O. - Redis is positioned as both a cache and a database due to its speed, data structures, and ability to handle complex data models through modules. - Redis Enterprise provides additional capabilities like durability, high availability, geo-distribution, security and multi-tenancy. - Modern data models in Redis modules include Streams, RediSearch, RedisGraph, RedisTimeSeries, RedisAI, RedisJSON and RedisBloom. - RedisInsight is

Leveraging Redis for System Monitoring by Adam McCormick of SBG - Redis Day S...Redis Labs

The document discusses how Sinclair Broadcast Group leverages Redis for system monitoring of its content delivery network. It operates 193 news stations with 10,000 active pages daily and millions in archive. New stories are posted every 15 seconds and must be visible across its 1,000+ targets within 1 minute. Redis is used to track performance across the multi-level CDN and ensure service level agreements are met with real-time resolution and alerting. It provides a black box view of the audience experience and can scale monitoring to all relevant pages within 30 seconds. Redis acts as a distributed data store to parallelize the monitoring task across the large scale of the network.

JSON in Redis - When to use RedisJSON by Jay Won of Coupang - Redis Day Seatt...Redis Labs

The document summarizes a presentation about when to use the RedisJSON data type. It discusses how Coupang uses Redis extensively for their ad platform. It then compares the performance and memory usage of storing JSON data as strings, hashes, or using the RedisJSON data type. Benchmark results show RedisJSON can provide better performance for retrieving and updating JSON fields compared to strings and hashes, though it uses more memory. The document recommends using RedisJSON for smaller JSON payloads after benchmarking and memory monitoring.

Highly Available Persistent Session Management Service by Mohamed Elmergawi o...Redis Labs

The document discusses the challenges of building a highly available persistent session management service. It describes Zulily's legacy architecture which lacked high availability and required manual intervention. A new architecture is proposed using Redis for persistent storage, Dynomite for real-time replication across data centers, and a connection pooling proxy to improve efficiency and distribute load. The architecture provides high availability through replication, reduces overhead through connection pooling, and handles failures through consistent hashing and health checks. It was tested through simulations and showed a failure rate of only 0.42% during outages.

Anatomy of a Redis Command by Madelyn Olson of Amazon Web Services - Redis Da...Redis Labs

The document describes the process that a Redis command follows from the client side to the server side. On the client side, the command is sent over the network to the Redis server. On the server side, the command is read from the kernel buffers, validated, executed by calling the relevant command handler, and the response is written back to the client over the network. The core functions involved on the server side are ReadQueryFromClient(), ProcessInputBuffer(), ProcessCommand(), Call(), and handleClientsWithPendingWrites(). Redis 6.0 introduced I/O threads to handle reads and writes in parallel for improved performance while still maintaining Redis' single-threaded processing model.

Building a Multi-dimensional Analytics Engine with RedisGraph by Matthew Goos...Redis Labs

This document discusses MDmetrix, a healthcare data intelligence company that uses RedisGraph to provide flexible analysis of hospital data. RedisGraph is a graph database that represents data as nodes and relationships and uses an adjacency matrix and linear algebra to query the graph. MDmetrix models its healthcare data as a property graph in RedisGraph to allow for complex queries across different data dimensions like patients, facilities, procedures and drugs. RedisGraph allows MDmetrix to query the data more easily than traditional OLAP cubes or relational databases due to the semi-structured and flexible nature of the graph model.

RediSearch 1.6 by Pieter Cailliau - Redis Day Bangalore 2020Redis Labs

RediSearch 1.6 includes a new low-level API that allows other Redis modules to embed RediSearch indexing capabilities. It also introduces index aliasing and several performance improvements such as forked thread garbage collection. Based on benchmarks, RediSearch 1.6 shows 48-73% better performance than version 1.4, particularly during high update rates where it maintains more stable read latencies.

RedisGraph 2.0 by Pieter Cailliau - Redis Day Bangalore 2020Redis Labs

RedisGraph 2.0 provides significant improvements including: - Full text search support through embedded RediSearch 1.6 enabling graph-aided search. - Support for returning full graph responses to enable better visualization. - Broad support for Cypher including triadic closure and new graph-aided search capabilities. - Performance improvements of up to 3.7x faster operations per second and 3.9x faster query times compared to RedisGraph v1.2. - Support for benchmarking including the LDBC benchmark.

RedisTimeSeries 1.2 by Pieter Cailliau - Redis Day Bangalore 2020Redis Labs

RedisTimeSeries is a time-series database that provides compression to reduce memory usage by up to 98% and improve performance. The RedisTimeSeries 1.2 release includes compression algorithms based on a Facebook paper that provide stable ingestion times independent of the number of data points. It also includes a reviewed API with performance improvements and clearer functionality. Performance testing showed ingestion throughput improved by 2-3% and query performance increased from 15-70% with the new release compared to the previous version.

RedisAI 0.9 by Sherin Thomas of Tensorwerk - Redis Day Bangalore 2020Redis Labs

This document summarizes RedisAI 0.9 and its capabilities for model deployment and benchmarking. It introduces RedisAI's new tensor data type and ability to deploy models to CPU and GPU. It then discusses AIBench, a tool developed to benchmark AI serving solutions like RedisAI, TensorFlow Serving, and REST APIs. The benchmarks show RedisAI providing 5.5x and 2.5x more inferences than REST APIs and TensorFlow Serving respectively, due to its data locality. The document concludes by mentioning RedisAI's integration with MLFlow for model deployment with a single command.

Rate-Limiting 30 Million requests by Vijay Lakshminarayanan and Girish Koundi...Redis Labs

The document discusses how Freshworks uses Redis Labs to rate limit 30 million API requests per day through their API gateway called Fluffy. Fluffy stores rate limit policies and maintains counters to track requests. Redis Labs allows Fluffy to easily scale to handle the high volume of requests by providing a fast, in-memory data store for managing rate limiting counters. The system was able to successfully rate limit 30 million requests per day with Redis Labs.

Three Pillars of Observability by Rajalakshmi Raji Srinivasan of Site24x7 Zoh...Redis Labs

Solving Complex Scaling Problems by Prashant Kumar and Abhishek Jain of Myntr...Redis Labs

Redis was used by Myntra to solve several complex scaling problems. It was used to build a scalable user segment service to support high read throughput of up to 5 million requests per minute with low latency. Redis allowed the service to scale beyond a single instance and included features like automatic backups and memory management. Redis also helped build a scalable mobile verification platform to reliably handle 100,000 requests per minute and scale to support higher future volumes. It was used as both a transient store and persistent backend. Finally, Redis locks helped build a scalable A/B testing platform by allowing experiments to be created and updated in an orderly concurrent fashion.