Introduction to Redis

REmote DIctionary Server
1/27/2017
2

1/27/2017 3
•Written in: C
•Main point: Blazing fast
•License: BSD
•Protocol: Telnet-like, binary safe
•Disk-backed in-memory database,

4
 in-memory data structure store
 Redis is an advanced key-value store, where keys can contain data
structures such as strings, hashes, lists, sets, and sorted sets. Supporting
a set of atomic operations on these data types.
Redis can persist data to the disk Redis is not only a key-value
store
 Can be used as Database, a Caching layer or a Message broker.
Redis is fast
Official webpage: http://redis.io
Source Code: https://github.com/antirez/redis
Online Demo: http://try.redis.io

 Redis is not a replacement for Relational Databases nor Document
Stores.
 It might be used complementary to a SQL relational store, and/or
NoSQL document store.
 Best Used: For rapidly changing data with a foreseeable database
size (should fit mostly in memory).
5
What Redis is not...

 Written in ANSI C by Salvatore Sanfilippo (@antirez).
 Works in most POSIX systems like Linux, BSD and OS X.
 Linux is the recommended
 No official support for Windows, but Microsoft develops and
maintains an open source Win-64 port of Redis*
 Redis is a single-threaded server, not designed to benefit from
multiple CPU cores. Several Redis instances can be launched to
scale out on several cores.
 All operations are atomic (no two commands can run at the same
time).
 It executes most commands in O(1) complexity and with minimal
lines of code.
1/27/2017 6

Caching
Pub/sub
Blocking/Delayed Queues
o Such as https://github.com/resque/resque
o http://techblog.netflix.com/2016/08/distributed-delay-queues-based-
on.html
Short lived items, with ttl
o Fraud detection
o User Sessions
o Request Filtering Service
Counting Reviews most bought items
Real time Analysis
Storing Unique items over time.
8
Use cases for Redis

 Speed is key
 More than just key-value pairs
 Redis is an in memory data store, so there are memory
limitations.
o An empty instance uses ~ 1MB of memory.
o 1 Million small Keys -> String Value pairs use ~ 100MB of
memory.
o 1 Million Keys -> Hash value, representing an object with 5
fields, use ~ 200 MB of memory.
 Dataset is not critical
1/27/2017 9

10
Redis Performance
https://redislabs.com/blog/the-proven-redis-performance#.WHKDFLYrJZ1

Comparing NoSQL Solutions In a Real-World Scenario:
Aerospike, Cassandra Open Source, Cassandra DataStax,
Couchbase and Redis Labs

Composed by Avalon Consulting, LLC
June 2015
1

11
Redis Throughput...
https://web.archive.org/web/20160304051043/http://techblog.netflix.com/2016/
01/dynomite-with-redis-on-aws-benchmarks_14.html

 Stat’s From Twitter(2014)
 Timeline Service for one datacenter using Hybrid List:
o ~40TB allocated heap
o ~30MM qps
o > 6,000 instances
 Use of BTree in one datacenter:
o ~65TB allocated heap
o ~9MM qps
o >4,000 instances
1/27/2017 12

1/27/2017 13
Redis Data Type Contains Read/write ability
String
Binary-safe strings (up to 512 MB),
Integers or Floating point values,
Bitmaps.
Operate on the whole string, parts,
increment/decrement the integers and floats,
get/set bits by position.
Hash
Unordered hash table of keys to
string values
Add, fetch, or remove individual ítems by key,
fetch the whole hash.
List Doubly linked list of strings
Push or pop items from both ends, trim based
on offsets, read individual or multiple items,
find or remove items by value.
Set
Unordered collection of unique
strings
Add, fetch, or remove individual items, check
membership, intersect, union, difference, fetch
random items.
Sorted Set
Ordered mapping of string members
to floating-point scores, ordered by
score
Add, fetch, or remove individual items, fetch
items based on score ranges or member value.
Geospatial
index
Sorted set implementation using
geospatial information as the score
Add, fetch or remove individual items, search
by coordinates and radius, calculate distance.
HyperLogLog
Probabilistic data structure to count
unique things using 12Kb of
memory
Add individual or multiple items, get the
cardinality.(http://antirez.com/news/75)

 Persistence
 Redis provides two mechanisms to deal with persistence: Redis database snapshots (RDB) and
append-only files (AOF).
 Replication
 A Redis instance known as the master, ensures that one or more instances kwown as the slaves, become
exact copies of the master. Clients can connect to the master or to the slaves. Slaves are read only by
default.
 Partitioning
 Breaking up data and distributing it across different hosts in a cluster.
 Can be implemented in different layers:
o Client: Partitioning on client-side code.
o Proxy: An extra layer that proxies all redis queries and performs partitioning (i.e.
Twemproxy,Dynomite).
o Query Router: instances will make sure to forward the query to the right node. (i.e. Redis Cluster).
o Redis Cluster is a mix between query routing and client side partitioning.
 Failover
o Manual
o Automatic with Redis Sentinel (for master-slave topology)
o Automatic with Redis Cluster (for cluster topology)
1/27/2017 14

15
Redis Persistence(Read more about it at
https://redis.io/topics/persistence )
• RDB
o performs point-in-time snapshots of your dataset at specified intervals
• AOF
o AOF persistence logs every write operation received by the serve
• If you wish, you can disable persistence at all, if you want your data to just exist as
long as the server is running
Master not persisting, Slave Persisting to disk.
• It is possible to combine both AOF and RDB in the same instance.

16
RDB advantages
o RDB is a very compact single-file point-in-time representation of your Redis data.
o RDB is very good for disaster recovery, being a single compact file can be transferred
to far data centers
o RDB maximizes Redis performances since the only work the Redis parent process
needs to do in order to persist is forking a child that will do all the rest. The parent
instance will never perform disk I/O or alike
o RDB allows faster restarts with big datasets compared to AOF.
RDB disadvantages
• RDB is NOT good if you need to minimize the chance of data loss in case Redis stops
working
o Fork() can be time consuming if the dataset is big, and may result in Redis to stop
serving clients for some millisecond or even for one second if the dataset is very big
and the CPU performance not great

o AOF advantages
• Using AOF Redis is much more durable: you can have different fsync policies: no fsync at all,
fsync every second, fsync at every query.
• The AOF log is an append only log, so there are no seeks, nor corruption problems if there is a
power outage
• Redis is able to automatically rewrite the AOF in background when it gets too big
• AOF contains a log of all the operations one after the other in an easy to understand and parse
format
o AOF Disadvantages
• AOF files are usually bigger than the equivalent RDB files for the same dataset
• AOF can be slower than RDB depending on the exact fsync policy. In general with fsync set
to every second performances are still very high.
1/27/2017 17

 Partitioning in Redis serves two main goals:
 It allows for much larger databases, using the sum of the memory of many computers. Without partitioning
you are limited to the amount of memory a single computer can support.
 It allows scaling the computational power to multiple cores and multiple computers, and the network
bandwidth to multiple computers and network adapters.
 Client side partitioning means that the clients directly select the right node where to
write or read a given key. Many Redis clients implement client side partitioning.e.g
JedisSharding
 Proxy assisted partitioning means that our clients send requests to a proxy that is able to
speak the Redis protocol, instead of sending requests directly to the right Redis instance.
The proxy will make sure to forward our request to the right Redis
proxy Twemproxy implements proxy assisted partitioning.
 Query routing means that you can send your query to a random instance, and the
instance will make sure to forward your query to the right node
1/27/2017Hystrix 18

 The key space is split into 16384 slots,
 Effectively setting an upper limit for the cluster size of 16384
master nodes (however the suggested max size of nodes is in
the order of ~ 1000 nodes).
 HASH_SLOT = CRC16(key) mod 16384
 When the cluster is stable, a single hash slot will be served by
a single node
 https://redis.io/topics/cluster-spec#keys-distribution-model
1/27/2017 20

 Redis Cluster supports the ability to add and remove nodes while
the cluster is running. (https://redis.io/topics/cluster-spec)
 Adding or removing a node is abstracted into the same operation:
moving a hash slot from one node to another.
o To add a new node to the cluster an empty node is added to the cluster
and some set of hash slots are moved from existing nodes to the new
node.
o To remove a node from the cluster the hash slots assigned to that node
are moved to other existing nodes.
o To rebalance the cluster a given set of hash slots are moved between
nodes.
1/27/2017 21

1/27/2017 22
Twemproxy (single)
 Twemproxy* works as a proxy between the clients and many Redis instances.
 Is able to automatically distribute data among different standalone Redis instances.
 Supports consistent hashing with different strategies and hashing functions.
 Multi-key commands and transactions are not supported.
Twemproxy (load balanced)
*Twemproxy is a project from Twitter and is not part of redis: https://github.com/twitter/twemproxy
**Illustrations from Redis Essentials book by Maxwell Dayvson Da Silva and Hugo Lopes Tavares

1/27/2017Dynomite 23
In the age of high scalability and big data,
Dynomite’s design goal is to turn those
single-server datastore solutions into peer-
to-peer, linearly scalable, clustered
systems while still preserving the native
client/server protocols of the datastores,
e.g., Redis protocol.

o Manual
o Automatic with Redis Sentinel (for master-slave topology)
o Automatic with Redis Cluster (for cluster topology)
1/27/2017 24

1/27/2017 26
Application
Client
Redis Slave
Sentinel
Redis
Redis Slave
Sentinel
Redis
Redis Master
Sentinel
Redis
Application
Client
Application
Client
Redis Slave
Redis
Sentinel
Writes And Reads
Writes And Reads Writes And Reads
Reads
Reads
Reads
Redis Master Redis Master
Sentinel Sentinel
Redis Redis
Reads
Reads

 Dynomite
 Twemproxy
1/27/2017 29

 Spring Data Redis project supports Jedis and Lettuce, two of
the most popular java clients for Redis.
 Other Java Redis client of note is Redisson.
1/27/2017 30

 A Request/Response server can be implemented so that it is able
to process new requests even if the client didn't already read the
old responses. This way it is possible to send multiple
commands to the server without waiting for the replies at all, and
finally read the replies in a single step.
 Lettuce client supports pipelining on redis cluster
 Jedis does not.
 Without pipelining 1.185238 seconds with pipelining 0.250783
seconds
 https://redis.io/topics/pipelining
1/27/2017 31

Introduction to Redis

Recommended

More Related Content

What's hot (20)

Similar to Introduction to Redis (20)

Recently uploaded (20)

Introduction to Redis