SlideShare a Scribd company logo
NoSQL
MongoDB and Redis as alternatives to
traditional RDBMS
Then...
...and now
*This thing weighs less than 50g
Meaning of NoSQL
1970 = We have no SQL
1980 = Know SQL
2000 = No SQL!
2005 = Not only SQL
2014 = No, SQL
(slide adapted from @markmadsen)
MongoDB
MongoDB
● it is the “new MySQL”
● Project started in 2007 by 10gen (now MongoDB Inc)
● Cross-platform, open-source
● 5th most used DBMS & most used Document Store*
(next DS CouchDB - 21st)
* According to db-engines.com as of Oct 2014
Characteristics
● “It's really a hybrid database with features from a few
different places.” (Gaetan Voyer-Perrault on Quora)
● Document Oriented but NO SCHEMA!
● Documents grouped in Collections
● Binary JSON (BSON) format
● Load Balancing (automated sharding, sharding key
can be user defined)
● Replication (Replica Sets)
● Automated failover
Characteristics - continued
● Primary and Secondary Indexes
● JavaScript for UDF
● MapReduce
● Capped Collections
● Aggregation Framework since 2.2
● Ad-hoc Query Support
Caveats
Generic performance tips
● Use 64-bit OS
● Lots of RAM, fast disks (was anyone expecting
something else?)
● ensure that at least indexes + working set fit in RAM
(db.stats(), db.<coll>.stats()) - if not, you might want to
try TokuMX
● Design for de-normalized data models
Generic performance tips
● Write-Concerns
● Shard early
● Fixed (or at least bounded) record size => better write
performance
● Use short attribute names (reduces index & data size,
OFC!)
● EXT4 or XFS
IRL
● virtualized server 8G RAM, 4 vCPU - no sharding, no
replica sets
● 100 inserts/s , 130M doc collection WITH secondary
index (avg doc size 0.6k)
● 20 inserts/s 3M doc collection WITH 18 secondary
indexes (avg doc size 10k)
Use Cases
● Logs
● Location Data (Mongo has built in Geospatial ops)
● Account and User Profiles
● Messaging
● (complex) Config Data
● http://www.mongodb.com/who-uses-mongodb (hint:
Expedia, Business Insider, The Weather Channel,
Foursquare, eBay)
Redis
Redis
● Salvatore Sanfilippo (@antirez)
● Started in 2009
● Key-Value Store
● 11th most used DBMS & most used KV Store* (next
KVS memcached - 19th)
● Sponsored by Pivotal (spinoff EMC/VMware)
* According to db-engines.com as of Oct 2014
Characteristics
● Holds all data in memory, persists on disk
● Data Models
○ Strings/Blobs/Bit-Maps (not really Bitmaps)
○ Hashtables
○ Linked Lists
○ Sets
○ Sorted Sets
● HyperLogLog (+2.8.9 - trade accuracy for memory)
● Master Slave Replication
● High Availability (through Sentinel)
Characteristics - continued
● Redis Cluster in works (not production ready yet) -
sharding
○ asynchronous replication
○ does not guarantee strong consistency (may ‘forget’ writes)
● AOF sync - default 2s
● Does not support secondary indexes
● Pub/Sub mode since 2.0
● Key expiry
● Server scripting with Lua
IRL
● virtualized server 4G RAM, 1vCPU
● +50k get/set per second (redis-benchmark)
● only 128 queries out of 1165550375 over 10ms
(0.00001%)
○ uptime_in_days:439
○ used_memory_human:424.09M
○ used_memory_peak_human:834.94M
○ total_connections_received:1352935
○ db0:keys=610884,expires=355397
Generic performance tips
● Use short key names (reduces data size, OFC!)
● You can create secondary indexes (but you have to
maintain them, e.g. using SET)
● You can have ad-hoc queries (actually is query) :
using SORT
Use Cases
● Cache
● IPSS/IPC
● Queue mechanisms (see e.g. Resque)
● Log/Task buffers
● Statistics and aggregation datastore
● (anywhere you use memcached)
● http://redis.io/topics/whos-using-redis (hint: Twitter,
GitHub, Snapchat, StackOverflow a.o.)
Recap
One size does NOT fit all!
Further reading
● Must read: http://blog.andreamostosi.name/big-
data/ (almost exhaustive list of all things NoSQL and
BigData)

More Related Content

What's hot (20)

COSCUP 2019 - The discussion between Knex.js and PostgreSQL
COSCUP 2019 - The discussion between Knex.js and PostgreSQLCOSCUP 2019 - The discussion between Knex.js and PostgreSQL
COSCUP 2019 - The discussion between Knex.js and PostgreSQL
Len Chang
 
Clusters With Glusterfs
Clusters With GlusterfsClusters With Glusterfs
Clusters With Glusterfs
Ontico
 
Gotszling mogo db-membase
Gotszling mogo db-membaseGotszling mogo db-membase
Gotszling mogo db-membase
GiltTech
 
Intro Couchdb
Intro CouchdbIntro Couchdb
Intro Couchdb
selvamanisampath
 
MongoDB Introduction - Document Oriented Nosql Database
MongoDB Introduction - Document Oriented Nosql DatabaseMongoDB Introduction - Document Oriented Nosql Database
MongoDB Introduction - Document Oriented Nosql Database
Sudhir Patil
 
Post-relational databases: What's wrong with web development? v3
Post-relational databases: What's wrong with web development? v3Post-relational databases: What's wrong with web development? v3
Post-relational databases: What's wrong with web development? v3
Dobrica Pavlinušić
 
Introduction to MongoDB (from Austin Code Camp)
Introduction to MongoDB (from Austin Code Camp)Introduction to MongoDB (from Austin Code Camp)
Introduction to MongoDB (from Austin Code Camp)
Chris Edwards
 
Putting rails and couch db on the cloud - Indicthreads cloud computing confe...
Putting rails and couch db on the cloud -  Indicthreads cloud computing confe...Putting rails and couch db on the cloud -  Indicthreads cloud computing confe...
Putting rails and couch db on the cloud - Indicthreads cloud computing confe...
IndicThreads
 
Steam Learn: An introduction to Redis
Steam Learn: An introduction to RedisSteam Learn: An introduction to Redis
Steam Learn: An introduction to Redis
inovia
 
Кирилл Шутемов aka “kas” - О Transparent Hugepages и Huge zero page
Кирилл Шутемов aka “kas” - О Transparent Hugepages и Huge zero pageКирилл Шутемов aka “kas” - О Transparent Hugepages и Huge zero page
Кирилл Шутемов aka “kas” - О Transparent Hugepages и Huge zero page
Minsk Linux User Group
 
110304 sha
110304 sha110304 sha
110304 sha
Shing AKA
 
Doing E-commerce Right – Magento on DigitalOcean
Doing E-commerce Right – Magento on DigitalOceanDoing E-commerce Right – Magento on DigitalOcean
Doing E-commerce Right – Magento on DigitalOcean
DigitalOcean
 
Level 101 for Presto: What is PrestoDB?
Level 101 for Presto: What is PrestoDB?Level 101 for Presto: What is PrestoDB?
Level 101 for Presto: What is PrestoDB?
Ali LeClerc
 
Redis IU
Redis IURedis IU
Redis IU
Isaiah Edem
 
Explore the Cosmos (DB) with .NET Core 2.0
Explore the Cosmos (DB) with .NET Core 2.0Explore the Cosmos (DB) with .NET Core 2.0
Explore the Cosmos (DB) with .NET Core 2.0
Jeremy Likness
 
Mysql to mongo
Mysql to mongoMysql to mongo
Mysql to mongo
Alex Sharp
 
CouchDB Talk JChris NYC
CouchDB Talk JChris NYCCouchDB Talk JChris NYC
CouchDB Talk JChris NYC
Chris Anderson
 
Introduction to new high performance storage engines in mongodb 3.0
Introduction to new high performance storage engines in mongodb 3.0Introduction to new high performance storage engines in mongodb 3.0
Introduction to new high performance storage engines in mongodb 3.0
Henrik Ingo
 
AMIS OOW Review - Deel 6 - Emiel Paasschens
AMIS OOW Review - Deel 6 - Emiel PaasschensAMIS OOW Review - Deel 6 - Emiel Paasschens
AMIS OOW Review - Deel 6 - Emiel Paasschens
Getting value from IoT, Integration and Data Analytics
 
JS App Architecture
JS App ArchitectureJS App Architecture
JS App Architecture
Corey Butler
 
COSCUP 2019 - The discussion between Knex.js and PostgreSQL
COSCUP 2019 - The discussion between Knex.js and PostgreSQLCOSCUP 2019 - The discussion between Knex.js and PostgreSQL
COSCUP 2019 - The discussion between Knex.js and PostgreSQL
Len Chang
 
Clusters With Glusterfs
Clusters With GlusterfsClusters With Glusterfs
Clusters With Glusterfs
Ontico
 
Gotszling mogo db-membase
Gotszling mogo db-membaseGotszling mogo db-membase
Gotszling mogo db-membase
GiltTech
 
MongoDB Introduction - Document Oriented Nosql Database
MongoDB Introduction - Document Oriented Nosql DatabaseMongoDB Introduction - Document Oriented Nosql Database
MongoDB Introduction - Document Oriented Nosql Database
Sudhir Patil
 
Post-relational databases: What's wrong with web development? v3
Post-relational databases: What's wrong with web development? v3Post-relational databases: What's wrong with web development? v3
Post-relational databases: What's wrong with web development? v3
Dobrica Pavlinušić
 
Introduction to MongoDB (from Austin Code Camp)
Introduction to MongoDB (from Austin Code Camp)Introduction to MongoDB (from Austin Code Camp)
Introduction to MongoDB (from Austin Code Camp)
Chris Edwards
 
Putting rails and couch db on the cloud - Indicthreads cloud computing confe...
Putting rails and couch db on the cloud -  Indicthreads cloud computing confe...Putting rails and couch db on the cloud -  Indicthreads cloud computing confe...
Putting rails and couch db on the cloud - Indicthreads cloud computing confe...
IndicThreads
 
Steam Learn: An introduction to Redis
Steam Learn: An introduction to RedisSteam Learn: An introduction to Redis
Steam Learn: An introduction to Redis
inovia
 
Кирилл Шутемов aka “kas” - О Transparent Hugepages и Huge zero page
Кирилл Шутемов aka “kas” - О Transparent Hugepages и Huge zero pageКирилл Шутемов aka “kas” - О Transparent Hugepages и Huge zero page
Кирилл Шутемов aka “kas” - О Transparent Hugepages и Huge zero page
Minsk Linux User Group
 
Doing E-commerce Right – Magento on DigitalOcean
Doing E-commerce Right – Magento on DigitalOceanDoing E-commerce Right – Magento on DigitalOcean
Doing E-commerce Right – Magento on DigitalOcean
DigitalOcean
 
Level 101 for Presto: What is PrestoDB?
Level 101 for Presto: What is PrestoDB?Level 101 for Presto: What is PrestoDB?
Level 101 for Presto: What is PrestoDB?
Ali LeClerc
 
Explore the Cosmos (DB) with .NET Core 2.0
Explore the Cosmos (DB) with .NET Core 2.0Explore the Cosmos (DB) with .NET Core 2.0
Explore the Cosmos (DB) with .NET Core 2.0
Jeremy Likness
 
Mysql to mongo
Mysql to mongoMysql to mongo
Mysql to mongo
Alex Sharp
 
CouchDB Talk JChris NYC
CouchDB Talk JChris NYCCouchDB Talk JChris NYC
CouchDB Talk JChris NYC
Chris Anderson
 
Introduction to new high performance storage engines in mongodb 3.0
Introduction to new high performance storage engines in mongodb 3.0Introduction to new high performance storage engines in mongodb 3.0
Introduction to new high performance storage engines in mongodb 3.0
Henrik Ingo
 
JS App Architecture
JS App ArchitectureJS App Architecture
JS App Architecture
Corey Butler
 

Viewers also liked (20)

lookbook 1
lookbook 1lookbook 1
lookbook 1
Fernando Castellanos
 
Give Dairy Farmers A Market
Give Dairy Farmers A MarketGive Dairy Farmers A Market
Give Dairy Farmers A Market
Catharinus Wierda
 
請敘述Cpu的執行過程或步驟
請敘述Cpu的執行過程或步驟請敘述Cpu的執行過程或步驟
請敘述Cpu的執行過程或步驟
Adolph YT
 
lookbook 3
lookbook 3 lookbook 3
lookbook 3
Fernando Castellanos
 
2015-OBDC-Organizational-Report
2015-OBDC-Organizational-Report2015-OBDC-Organizational-Report
2015-OBDC-Organizational-Report
Rachel Aoanan
 
Game sense
Game senseGame sense
Game sense
17280495
 
Power poin programacion
Power poin programacionPower poin programacion
Power poin programacion
Patyy Nolasco
 
society and Daily Life in Mogul India
society and Daily Life in Mogul Indiasociety and Daily Life in Mogul India
society and Daily Life in Mogul India
mariacardenas02
 
Deconstructing Lambda architectures
Deconstructing Lambda architecturesDeconstructing Lambda architectures
Deconstructing Lambda architectures
Felix Crisan
 
Mashing the data
Mashing the dataMashing the data
Mashing the data
Felix Crisan
 
Presentation for the first Bucharest Big data meetup
Presentation for the first Bucharest Big data meetupPresentation for the first Bucharest Big data meetup
Presentation for the first Bucharest Big data meetup
Felix Crisan
 
Estudio de caso
Estudio de casoEstudio de caso
Estudio de caso
veronica pallarozo
 
latihan un smp
latihan un smplatihan un smp
latihan un smp
Astried Wulandari
 
Grafico diario del dax perfomance index para el 10 02-2012
Grafico diario del dax perfomance index para el 10 02-2012Grafico diario del dax perfomance index para el 10 02-2012
Grafico diario del dax perfomance index para el 10 02-2012
Experiencia Trading
 
U.S. Navy Twitter Business Rules 2015
U.S. Navy Twitter Business Rules 2015U.S. Navy Twitter Business Rules 2015
U.S. Navy Twitter Business Rules 2015
US Navy Social Media
 
Mi Escuela
Mi Escuela Mi Escuela
Mi Escuela
Yuciri Estrada
 
оценување на вештините на учениците за решавање проблеми
оценување на вештините на учениците за решавање проблемиоценување на вештините на учениците за решавање проблеми
оценување на вештините на учениците за решавање проблеми
Anica Petkoska
 
Dreamforce 2015 - 4 days in 4 minutes
Dreamforce 2015 - 4 days in 4 minutesDreamforce 2015 - 4 days in 4 minutes
Dreamforce 2015 - 4 days in 4 minutes
Desynit
 
Partes de la computadora sjb
Partes de la computadora sjbPartes de la computadora sjb
Partes de la computadora sjb
Jazdaly Meza
 
請敘述Cpu的執行過程或步驟
請敘述Cpu的執行過程或步驟請敘述Cpu的執行過程或步驟
請敘述Cpu的執行過程或步驟
Adolph YT
 
2015-OBDC-Organizational-Report
2015-OBDC-Organizational-Report2015-OBDC-Organizational-Report
2015-OBDC-Organizational-Report
Rachel Aoanan
 
Game sense
Game senseGame sense
Game sense
17280495
 
Power poin programacion
Power poin programacionPower poin programacion
Power poin programacion
Patyy Nolasco
 
society and Daily Life in Mogul India
society and Daily Life in Mogul Indiasociety and Daily Life in Mogul India
society and Daily Life in Mogul India
mariacardenas02
 
Deconstructing Lambda architectures
Deconstructing Lambda architecturesDeconstructing Lambda architectures
Deconstructing Lambda architectures
Felix Crisan
 
Presentation for the first Bucharest Big data meetup
Presentation for the first Bucharest Big data meetupPresentation for the first Bucharest Big data meetup
Presentation for the first Bucharest Big data meetup
Felix Crisan
 
Grafico diario del dax perfomance index para el 10 02-2012
Grafico diario del dax perfomance index para el 10 02-2012Grafico diario del dax perfomance index para el 10 02-2012
Grafico diario del dax perfomance index para el 10 02-2012
Experiencia Trading
 
U.S. Navy Twitter Business Rules 2015
U.S. Navy Twitter Business Rules 2015U.S. Navy Twitter Business Rules 2015
U.S. Navy Twitter Business Rules 2015
US Navy Social Media
 
оценување на вештините на учениците за решавање проблеми
оценување на вештините на учениците за решавање проблемиоценување на вештините на учениците за решавање проблеми
оценување на вештините на учениците за решавање проблеми
Anica Petkoska
 
Dreamforce 2015 - 4 days in 4 minutes
Dreamforce 2015 - 4 days in 4 minutesDreamforce 2015 - 4 days in 4 minutes
Dreamforce 2015 - 4 days in 4 minutes
Desynit
 
Partes de la computadora sjb
Partes de la computadora sjbPartes de la computadora sjb
Partes de la computadora sjb
Jazdaly Meza
 

Similar to NoSQL solutions (20)

002 Database-Engines.pptx
002 Database-Engines.pptx002 Database-Engines.pptx
002 Database-Engines.pptx
Irfan Ullah Khan
 
Postgres-XC as a Key Value Store Compared To MongoDB
Postgres-XC as a Key Value Store Compared To MongoDBPostgres-XC as a Key Value Store Compared To MongoDB
Postgres-XC as a Key Value Store Compared To MongoDB
Mason Sharp
 
MongoDB 2.4 and spring data
MongoDB 2.4 and spring dataMongoDB 2.4 and spring data
MongoDB 2.4 and spring data
Jimmy Ray
 
MongoDB: Advantages of an Open Source NoSQL Database
MongoDB: Advantages of an Open Source NoSQL DatabaseMongoDB: Advantages of an Open Source NoSQL Database
MongoDB: Advantages of an Open Source NoSQL Database
FITC
 
No sql bigdata and postgresql
No sql bigdata and postgresqlNo sql bigdata and postgresql
No sql bigdata and postgresql
Zaid Shabbir
 
Scaling up and accelerating Drupal 8 with NoSQL
Scaling up and accelerating Drupal 8 with NoSQLScaling up and accelerating Drupal 8 with NoSQL
Scaling up and accelerating Drupal 8 with NoSQL
OSInet
 
BlackRay - The open Source Data Engine
BlackRay - The open Source Data EngineBlackRay - The open Source Data Engine
BlackRay - The open Source Data Engine
fschupp
 
No SQL Technologies
No SQL TechnologiesNo SQL Technologies
No SQL Technologies
Cris Holdorph
 
Ukoug 2011 mysql_arch_for_orcl_dba
Ukoug 2011 mysql_arch_for_orcl_dbaUkoug 2011 mysql_arch_for_orcl_dba
Ukoug 2011 mysql_arch_for_orcl_dba
orablue11
 
New paradigms
New paradigmsNew paradigms
New paradigms
Borja A. Espejo García
 
MongoDB NoSQL - Developer Guide
MongoDB NoSQL - Developer GuideMongoDB NoSQL - Developer Guide
MongoDB NoSQL - Developer Guide
Shiv K Sah
 
Mongo db
Mongo dbMongo db
Mongo db
Swecha | స్వేచ్ఛ
 
Postgres-XC Write Scalable PostgreSQL Cluster
Postgres-XC Write Scalable PostgreSQL ClusterPostgres-XC Write Scalable PostgreSQL Cluster
Postgres-XC Write Scalable PostgreSQL Cluster
Mason Sharp
 
Mongodb (1)
Mongodb (1)Mongodb (1)
Mongodb (1)
Deepak Kumar
 
Blackray @ SAPO CodeBits 2009
Blackray @ SAPO CodeBits 2009Blackray @ SAPO CodeBits 2009
Blackray @ SAPO CodeBits 2009
fschupp
 
Next.js with drupal, the good parts
Next.js with drupal, the good partsNext.js with drupal, the good parts
Next.js with drupal, the good parts
Taller Negócio Digitais
 
Lean and mean MongoDB
Lean and mean MongoDBLean and mean MongoDB
Lean and mean MongoDB
Oleg Podsechin
 
kranonit S06E01 Игорь Цинько: High load
kranonit S06E01 Игорь Цинько: High loadkranonit S06E01 Игорь Цинько: High load
kranonit S06E01 Игорь Цинько: High load
Krivoy Rog IT Community
 
Mongodb open data day 2014
Mongodb open data day 2014Mongodb open data day 2014
Mongodb open data day 2014
David Green
 
ArangoDB
ArangoDBArangoDB
ArangoDB
ArangoDB Database
 
Postgres-XC as a Key Value Store Compared To MongoDB
Postgres-XC as a Key Value Store Compared To MongoDBPostgres-XC as a Key Value Store Compared To MongoDB
Postgres-XC as a Key Value Store Compared To MongoDB
Mason Sharp
 
MongoDB 2.4 and spring data
MongoDB 2.4 and spring dataMongoDB 2.4 and spring data
MongoDB 2.4 and spring data
Jimmy Ray
 
MongoDB: Advantages of an Open Source NoSQL Database
MongoDB: Advantages of an Open Source NoSQL DatabaseMongoDB: Advantages of an Open Source NoSQL Database
MongoDB: Advantages of an Open Source NoSQL Database
FITC
 
No sql bigdata and postgresql
No sql bigdata and postgresqlNo sql bigdata and postgresql
No sql bigdata and postgresql
Zaid Shabbir
 
Scaling up and accelerating Drupal 8 with NoSQL
Scaling up and accelerating Drupal 8 with NoSQLScaling up and accelerating Drupal 8 with NoSQL
Scaling up and accelerating Drupal 8 with NoSQL
OSInet
 
BlackRay - The open Source Data Engine
BlackRay - The open Source Data EngineBlackRay - The open Source Data Engine
BlackRay - The open Source Data Engine
fschupp
 
Ukoug 2011 mysql_arch_for_orcl_dba
Ukoug 2011 mysql_arch_for_orcl_dbaUkoug 2011 mysql_arch_for_orcl_dba
Ukoug 2011 mysql_arch_for_orcl_dba
orablue11
 
MongoDB NoSQL - Developer Guide
MongoDB NoSQL - Developer GuideMongoDB NoSQL - Developer Guide
MongoDB NoSQL - Developer Guide
Shiv K Sah
 
Postgres-XC Write Scalable PostgreSQL Cluster
Postgres-XC Write Scalable PostgreSQL ClusterPostgres-XC Write Scalable PostgreSQL Cluster
Postgres-XC Write Scalable PostgreSQL Cluster
Mason Sharp
 
Blackray @ SAPO CodeBits 2009
Blackray @ SAPO CodeBits 2009Blackray @ SAPO CodeBits 2009
Blackray @ SAPO CodeBits 2009
fschupp
 
kranonit S06E01 Игорь Цинько: High load
kranonit S06E01 Игорь Цинько: High loadkranonit S06E01 Игорь Цинько: High load
kranonit S06E01 Игорь Цинько: High load
Krivoy Rog IT Community
 
Mongodb open data day 2014
Mongodb open data day 2014Mongodb open data day 2014
Mongodb open data day 2014
David Green
 

More from Felix Crisan (12)

Big data uservices
Big data uservicesBig data uservices
Big data uservices
Felix Crisan
 
Bitcoin:Next
Bitcoin:NextBitcoin:Next
Bitcoin:Next
Felix Crisan
 
BigData in BlockChains
BigData in BlockChainsBigData in BlockChains
BigData in BlockChains
Felix Crisan
 
Lightning Network
Lightning  NetworkLightning  Network
Lightning Network
Felix Crisan
 
Smart contracts using web3.js
Smart contracts using web3.jsSmart contracts using web3.js
Smart contracts using web3.js
Felix Crisan
 
Smart contracts in Solidity
Smart contracts in SoliditySmart contracts in Solidity
Smart contracts in Solidity
Felix Crisan
 
Big(data) in block(chains)
Big(data) in block(chains)Big(data) in block(chains)
Big(data) in block(chains)
Felix Crisan
 
Enablers for o commerce
Enablers for o commerceEnablers for o commerce
Enablers for o commerce
Felix Crisan
 
mcommad
mcommadmcommad
mcommad
Felix Crisan
 
402 @ Mobile next
402 @ Mobile next402 @ Mobile next
402 @ Mobile next
Felix Crisan
 
Data analysis with Pandas and Spark
Data analysis with Pandas and SparkData analysis with Pandas and Spark
Data analysis with Pandas and Spark
Felix Crisan
 
TCP/IP of money
TCP/IP of moneyTCP/IP of money
TCP/IP of money
Felix Crisan
 
Big data uservices
Big data uservicesBig data uservices
Big data uservices
Felix Crisan
 
BigData in BlockChains
BigData in BlockChainsBigData in BlockChains
BigData in BlockChains
Felix Crisan
 
Smart contracts using web3.js
Smart contracts using web3.jsSmart contracts using web3.js
Smart contracts using web3.js
Felix Crisan
 
Smart contracts in Solidity
Smart contracts in SoliditySmart contracts in Solidity
Smart contracts in Solidity
Felix Crisan
 
Big(data) in block(chains)
Big(data) in block(chains)Big(data) in block(chains)
Big(data) in block(chains)
Felix Crisan
 
Enablers for o commerce
Enablers for o commerceEnablers for o commerce
Enablers for o commerce
Felix Crisan
 
Data analysis with Pandas and Spark
Data analysis with Pandas and SparkData analysis with Pandas and Spark
Data analysis with Pandas and Spark
Felix Crisan
 

Recently uploaded (20)

Semantic Cultivators : The Critical Future Role to Enable AI
Semantic Cultivators : The Critical Future Role to Enable AISemantic Cultivators : The Critical Future Role to Enable AI
Semantic Cultivators : The Critical Future Role to Enable AI
artmondano
 
Increasing Retail Store Efficiency How can Planograms Save Time and Money.pptx
Increasing Retail Store Efficiency How can Planograms Save Time and Money.pptxIncreasing Retail Store Efficiency How can Planograms Save Time and Money.pptx
Increasing Retail Store Efficiency How can Planograms Save Time and Money.pptx
Anoop Ashok
 
Massive Power Outage Hits Spain, Portugal, and France: Causes, Impact, and On...
Massive Power Outage Hits Spain, Portugal, and France: Causes, Impact, and On...Massive Power Outage Hits Spain, Portugal, and France: Causes, Impact, and On...
Massive Power Outage Hits Spain, Portugal, and France: Causes, Impact, and On...
Aqusag Technologies
 
Big Data Analytics Quick Research Guide by Arthur Morgan
Big Data Analytics Quick Research Guide by Arthur MorganBig Data Analytics Quick Research Guide by Arthur Morgan
Big Data Analytics Quick Research Guide by Arthur Morgan
Arthur Morgan
 
tecnologias de las primeras civilizaciones.pdf
tecnologias de las primeras civilizaciones.pdftecnologias de las primeras civilizaciones.pdf
tecnologias de las primeras civilizaciones.pdf
fjgm517
 
2025-05-Q4-2024-Investor-Presentation.pptx
2025-05-Q4-2024-Investor-Presentation.pptx2025-05-Q4-2024-Investor-Presentation.pptx
2025-05-Q4-2024-Investor-Presentation.pptx
Samuele Fogagnolo
 
TrsLabs - Fintech Product & Business Consulting
TrsLabs - Fintech Product & Business ConsultingTrsLabs - Fintech Product & Business Consulting
TrsLabs - Fintech Product & Business Consulting
Trs Labs
 
Splunk Security Update | Public Sector Summit Germany 2025
Splunk Security Update | Public Sector Summit Germany 2025Splunk Security Update | Public Sector Summit Germany 2025
Splunk Security Update | Public Sector Summit Germany 2025
Splunk
 
HCL Nomad Web – Best Practices and Managing Multiuser Environments
HCL Nomad Web – Best Practices and Managing Multiuser EnvironmentsHCL Nomad Web – Best Practices and Managing Multiuser Environments
HCL Nomad Web – Best Practices and Managing Multiuser Environments
panagenda
 
Procurement Insights Cost To Value Guide.pptx
Procurement Insights Cost To Value Guide.pptxProcurement Insights Cost To Value Guide.pptx
Procurement Insights Cost To Value Guide.pptx
Jon Hansen
 
How analogue intelligence complements AI
How analogue intelligence complements AIHow analogue intelligence complements AI
How analogue intelligence complements AI
Paul Rowe
 
Mobile App Development Company in Saudi Arabia
Mobile App Development Company in Saudi ArabiaMobile App Development Company in Saudi Arabia
Mobile App Development Company in Saudi Arabia
Steve Jonas
 
AI Changes Everything – Talk at Cardiff Metropolitan University, 29th April 2...
AI Changes Everything – Talk at Cardiff Metropolitan University, 29th April 2...AI Changes Everything – Talk at Cardiff Metropolitan University, 29th April 2...
AI Changes Everything – Talk at Cardiff Metropolitan University, 29th April 2...
Alan Dix
 
Dev Dives: Automate and orchestrate your processes with UiPath Maestro
Dev Dives: Automate and orchestrate your processes with UiPath MaestroDev Dives: Automate and orchestrate your processes with UiPath Maestro
Dev Dives: Automate and orchestrate your processes with UiPath Maestro
UiPathCommunity
 
Andrew Marnell: Transforming Business Strategy Through Data-Driven Insights
Andrew Marnell: Transforming Business Strategy Through Data-Driven InsightsAndrew Marnell: Transforming Business Strategy Through Data-Driven Insights
Andrew Marnell: Transforming Business Strategy Through Data-Driven Insights
Andrew Marnell
 
DevOpsDays Atlanta 2025 - Building 10x Development Organizations.pptx
DevOpsDays Atlanta 2025 - Building 10x Development Organizations.pptxDevOpsDays Atlanta 2025 - Building 10x Development Organizations.pptx
DevOpsDays Atlanta 2025 - Building 10x Development Organizations.pptx
Justin Reock
 
Electronic_Mail_Attacks-1-35.pdf by xploit
Electronic_Mail_Attacks-1-35.pdf by xploitElectronic_Mail_Attacks-1-35.pdf by xploit
Electronic_Mail_Attacks-1-35.pdf by xploit
niftliyevhuseyn
 
Complete Guide to Advanced Logistics Management Software in Riyadh.pdf
Complete Guide to Advanced Logistics Management Software in Riyadh.pdfComplete Guide to Advanced Logistics Management Software in Riyadh.pdf
Complete Guide to Advanced Logistics Management Software in Riyadh.pdf
Software Company
 
HCL Nomad Web – Best Practices und Verwaltung von Multiuser-Umgebungen
HCL Nomad Web – Best Practices und Verwaltung von Multiuser-UmgebungenHCL Nomad Web – Best Practices und Verwaltung von Multiuser-Umgebungen
HCL Nomad Web – Best Practices und Verwaltung von Multiuser-Umgebungen
panagenda
 
Linux Professional Institute LPIC-1 Exam.pdf
Linux Professional Institute LPIC-1 Exam.pdfLinux Professional Institute LPIC-1 Exam.pdf
Linux Professional Institute LPIC-1 Exam.pdf
RHCSA Guru
 
Semantic Cultivators : The Critical Future Role to Enable AI
Semantic Cultivators : The Critical Future Role to Enable AISemantic Cultivators : The Critical Future Role to Enable AI
Semantic Cultivators : The Critical Future Role to Enable AI
artmondano
 
Increasing Retail Store Efficiency How can Planograms Save Time and Money.pptx
Increasing Retail Store Efficiency How can Planograms Save Time and Money.pptxIncreasing Retail Store Efficiency How can Planograms Save Time and Money.pptx
Increasing Retail Store Efficiency How can Planograms Save Time and Money.pptx
Anoop Ashok
 
Massive Power Outage Hits Spain, Portugal, and France: Causes, Impact, and On...
Massive Power Outage Hits Spain, Portugal, and France: Causes, Impact, and On...Massive Power Outage Hits Spain, Portugal, and France: Causes, Impact, and On...
Massive Power Outage Hits Spain, Portugal, and France: Causes, Impact, and On...
Aqusag Technologies
 
Big Data Analytics Quick Research Guide by Arthur Morgan
Big Data Analytics Quick Research Guide by Arthur MorganBig Data Analytics Quick Research Guide by Arthur Morgan
Big Data Analytics Quick Research Guide by Arthur Morgan
Arthur Morgan
 
tecnologias de las primeras civilizaciones.pdf
tecnologias de las primeras civilizaciones.pdftecnologias de las primeras civilizaciones.pdf
tecnologias de las primeras civilizaciones.pdf
fjgm517
 
2025-05-Q4-2024-Investor-Presentation.pptx
2025-05-Q4-2024-Investor-Presentation.pptx2025-05-Q4-2024-Investor-Presentation.pptx
2025-05-Q4-2024-Investor-Presentation.pptx
Samuele Fogagnolo
 
TrsLabs - Fintech Product & Business Consulting
TrsLabs - Fintech Product & Business ConsultingTrsLabs - Fintech Product & Business Consulting
TrsLabs - Fintech Product & Business Consulting
Trs Labs
 
Splunk Security Update | Public Sector Summit Germany 2025
Splunk Security Update | Public Sector Summit Germany 2025Splunk Security Update | Public Sector Summit Germany 2025
Splunk Security Update | Public Sector Summit Germany 2025
Splunk
 
HCL Nomad Web – Best Practices and Managing Multiuser Environments
HCL Nomad Web – Best Practices and Managing Multiuser EnvironmentsHCL Nomad Web – Best Practices and Managing Multiuser Environments
HCL Nomad Web – Best Practices and Managing Multiuser Environments
panagenda
 
Procurement Insights Cost To Value Guide.pptx
Procurement Insights Cost To Value Guide.pptxProcurement Insights Cost To Value Guide.pptx
Procurement Insights Cost To Value Guide.pptx
Jon Hansen
 
How analogue intelligence complements AI
How analogue intelligence complements AIHow analogue intelligence complements AI
How analogue intelligence complements AI
Paul Rowe
 
Mobile App Development Company in Saudi Arabia
Mobile App Development Company in Saudi ArabiaMobile App Development Company in Saudi Arabia
Mobile App Development Company in Saudi Arabia
Steve Jonas
 
AI Changes Everything – Talk at Cardiff Metropolitan University, 29th April 2...
AI Changes Everything – Talk at Cardiff Metropolitan University, 29th April 2...AI Changes Everything – Talk at Cardiff Metropolitan University, 29th April 2...
AI Changes Everything – Talk at Cardiff Metropolitan University, 29th April 2...
Alan Dix
 
Dev Dives: Automate and orchestrate your processes with UiPath Maestro
Dev Dives: Automate and orchestrate your processes with UiPath MaestroDev Dives: Automate and orchestrate your processes with UiPath Maestro
Dev Dives: Automate and orchestrate your processes with UiPath Maestro
UiPathCommunity
 
Andrew Marnell: Transforming Business Strategy Through Data-Driven Insights
Andrew Marnell: Transforming Business Strategy Through Data-Driven InsightsAndrew Marnell: Transforming Business Strategy Through Data-Driven Insights
Andrew Marnell: Transforming Business Strategy Through Data-Driven Insights
Andrew Marnell
 
DevOpsDays Atlanta 2025 - Building 10x Development Organizations.pptx
DevOpsDays Atlanta 2025 - Building 10x Development Organizations.pptxDevOpsDays Atlanta 2025 - Building 10x Development Organizations.pptx
DevOpsDays Atlanta 2025 - Building 10x Development Organizations.pptx
Justin Reock
 
Electronic_Mail_Attacks-1-35.pdf by xploit
Electronic_Mail_Attacks-1-35.pdf by xploitElectronic_Mail_Attacks-1-35.pdf by xploit
Electronic_Mail_Attacks-1-35.pdf by xploit
niftliyevhuseyn
 
Complete Guide to Advanced Logistics Management Software in Riyadh.pdf
Complete Guide to Advanced Logistics Management Software in Riyadh.pdfComplete Guide to Advanced Logistics Management Software in Riyadh.pdf
Complete Guide to Advanced Logistics Management Software in Riyadh.pdf
Software Company
 
HCL Nomad Web – Best Practices und Verwaltung von Multiuser-Umgebungen
HCL Nomad Web – Best Practices und Verwaltung von Multiuser-UmgebungenHCL Nomad Web – Best Practices und Verwaltung von Multiuser-Umgebungen
HCL Nomad Web – Best Practices und Verwaltung von Multiuser-Umgebungen
panagenda
 
Linux Professional Institute LPIC-1 Exam.pdf
Linux Professional Institute LPIC-1 Exam.pdfLinux Professional Institute LPIC-1 Exam.pdf
Linux Professional Institute LPIC-1 Exam.pdf
RHCSA Guru
 

NoSQL solutions

  • 1. NoSQL MongoDB and Redis as alternatives to traditional RDBMS
  • 3. ...and now *This thing weighs less than 50g
  • 4. Meaning of NoSQL 1970 = We have no SQL 1980 = Know SQL 2000 = No SQL! 2005 = Not only SQL 2014 = No, SQL (slide adapted from @markmadsen)
  • 6. MongoDB ● it is the “new MySQL” ● Project started in 2007 by 10gen (now MongoDB Inc) ● Cross-platform, open-source ● 5th most used DBMS & most used Document Store* (next DS CouchDB - 21st) * According to db-engines.com as of Oct 2014
  • 7. Characteristics ● “It's really a hybrid database with features from a few different places.” (Gaetan Voyer-Perrault on Quora) ● Document Oriented but NO SCHEMA! ● Documents grouped in Collections ● Binary JSON (BSON) format ● Load Balancing (automated sharding, sharding key can be user defined) ● Replication (Replica Sets) ● Automated failover
  • 8. Characteristics - continued ● Primary and Secondary Indexes ● JavaScript for UDF ● MapReduce ● Capped Collections ● Aggregation Framework since 2.2 ● Ad-hoc Query Support
  • 10. Generic performance tips ● Use 64-bit OS ● Lots of RAM, fast disks (was anyone expecting something else?) ● ensure that at least indexes + working set fit in RAM (db.stats(), db.<coll>.stats()) - if not, you might want to try TokuMX ● Design for de-normalized data models
  • 11. Generic performance tips ● Write-Concerns ● Shard early ● Fixed (or at least bounded) record size => better write performance ● Use short attribute names (reduces index & data size, OFC!) ● EXT4 or XFS
  • 12. IRL ● virtualized server 8G RAM, 4 vCPU - no sharding, no replica sets ● 100 inserts/s , 130M doc collection WITH secondary index (avg doc size 0.6k) ● 20 inserts/s 3M doc collection WITH 18 secondary indexes (avg doc size 10k)
  • 13. Use Cases ● Logs ● Location Data (Mongo has built in Geospatial ops) ● Account and User Profiles ● Messaging ● (complex) Config Data ● http://www.mongodb.com/who-uses-mongodb (hint: Expedia, Business Insider, The Weather Channel, Foursquare, eBay)
  • 14. Redis
  • 15. Redis ● Salvatore Sanfilippo (@antirez) ● Started in 2009 ● Key-Value Store ● 11th most used DBMS & most used KV Store* (next KVS memcached - 19th) ● Sponsored by Pivotal (spinoff EMC/VMware) * According to db-engines.com as of Oct 2014
  • 16. Characteristics ● Holds all data in memory, persists on disk ● Data Models ○ Strings/Blobs/Bit-Maps (not really Bitmaps) ○ Hashtables ○ Linked Lists ○ Sets ○ Sorted Sets ● HyperLogLog (+2.8.9 - trade accuracy for memory) ● Master Slave Replication ● High Availability (through Sentinel)
  • 17. Characteristics - continued ● Redis Cluster in works (not production ready yet) - sharding ○ asynchronous replication ○ does not guarantee strong consistency (may ‘forget’ writes) ● AOF sync - default 2s ● Does not support secondary indexes ● Pub/Sub mode since 2.0 ● Key expiry ● Server scripting with Lua
  • 18. IRL ● virtualized server 4G RAM, 1vCPU ● +50k get/set per second (redis-benchmark) ● only 128 queries out of 1165550375 over 10ms (0.00001%) ○ uptime_in_days:439 ○ used_memory_human:424.09M ○ used_memory_peak_human:834.94M ○ total_connections_received:1352935 ○ db0:keys=610884,expires=355397
  • 19. Generic performance tips ● Use short key names (reduces data size, OFC!) ● You can create secondary indexes (but you have to maintain them, e.g. using SET) ● You can have ad-hoc queries (actually is query) : using SORT
  • 20. Use Cases ● Cache ● IPSS/IPC ● Queue mechanisms (see e.g. Resque) ● Log/Task buffers ● Statistics and aggregation datastore ● (anywhere you use memcached) ● http://redis.io/topics/whos-using-redis (hint: Twitter, GitHub, Snapchat, StackOverflow a.o.)
  • 21. Recap One size does NOT fit all!
  • 22. Further reading ● Must read: http://blog.andreamostosi.name/big- data/ (almost exhaustive list of all things NoSQL and BigData)