NOSQL Data Management

Introduction
• It is a non-relational database
• Varies from traditional relational database system.
• It is provided for distributed data stores where there is a need for large
scale of data storing.
• Eg. Google and Facebook are collecting terabytes of data daily for their
users.
• This database does not require fixed schema, avoid join operations, and
scale data horizontally.
• In a NOSQL database, tables are stored in ASCII files, with each tuple
represented by a row and fields separated with tabs.
• NOSQL doesn’t use SQL as a Query Language.
• NOSQL databases are still in the development stages and are going
through a lot of changes.
• Software developers who work on database have a mixed opinion
about NOSQL.
• Some find it useful, while others point out flaws in it.
• Some are uncertain and believe that its just another hyped
technology that will eventually vanish due to its immaturity.
• Highlights:
• Introduction to NOSQL
• Aggregate data models
• Key-value,Column-orient, document and graph model.
• Explain the concept of relationships in NOSQL and schema-less databases.
• Materialized views as well as distribution model
• Sharding
Introduction to NOSQL
• Lot of data storage options in Market and many more still coming up as the
nature of data, platforms, user requirements, architectures and processes
are constantly changing.
• We are living in the era of Big Data and are in search of ways of handling it.
• This has given impetus to, since 2009, the need for creating schema –free
databases that can handle large amounts of data.
• These databases are scalable, enable availability of user, support
replication and are distributed and possibly open source.
• One such database is NOSQL.
• Before you develop applications that can interact with NOSQL databases,
you should first understand the need for maintaining a separation between
data management and data storage in these databases.
• NOSQL mainly focuses on high performance scalable data storage and
provides low-level access to the data management layer.
• This allows data management tasks to be created easily in any
programming language.
Interaction of Layers in NOSQL
Why NOSQL?
• Modern NOSQL solutions can tackle applications that require a high
degree of scalability, data distribution and continuous availability.
• We should also understand that relational databases have been used
for transactional processing in organizations for a very long time.
• RDBMS is supported by several tools and a large trained workforce
that implements and maintains them.
• Hence, the need to find an alternative to RDBMS is technical,
functional and perhaps financial.
 Benefits and Challenges of NOSQL
Benefits of NOSQL Databases Challenges against NOSQL Databases
Scale(Horizontal) ACID transaction(Atomicity, Consistency, Isolation, Durability)
Simple data model(fewer joins) Cannot use SQL
Streaming/Volume Ecosystem/tools/add-ons
Reliability Cannot perform searches
Schema-less(no modelling or prototyping) Data loss
Rapid Development No referential integrity
Flexible as it can handle semi-structured, unstructured and Lack of availability of expertise
structured data
Cheaper than the relational database
Creates a caching layer
Wide data type variety
Uses large binary objects for storing large data
Bulk upload
Graphs
Lower administration
Distributed Storage
Characteristics of NOSQL
• NOSQL databases do not use SQL; however some of them have query language.
• When the creators of NOSQL chose the name NOSQL, they meant to
communicate the fact that here is database query language that does not follow
the principles of RDBMS.
• Another important characteristics of NOSQL databases is that they are generally
open-source projects.
• The term NOSQL is frequently applied to an open-source phenomenon.
• Most NOSQL databases are driven by the need to run on clusters.
• Eg. ACID transactions are used to maintain consistency across the whole
database, which is alos a characteristics of relational databases.
• NOSQL databases offer a range of options for consistency and distribution.
• You are freely allowed to add fields to database records without having to define
any changes in the structure because NOSQL database operate without a schema.
• This comes particularly useful while dealing with non-uniform data and
custom fields.
• Relational databases as one option for data storage.
• This point of view is often referred to as polyglot persistence, and simply
means using different data stores for different circumstances.
• We need to understand the nature of data we are storing and how we want
to manipulate it, instead of picking up relational database just because
everyone is doing it.
• As a result, most organizations will have a mix of data storage technologies
for different circumstances.
• However, Big data has created an opportunity for people to think with a
new perspective on their data storage needs.
• Some development teams see in NOSQL database the opportunity of
improving their productivity by simplifying database access.
History of NOSQL
• The NOSQL database was invented by an engineer name carlo strozzi in
1998.
• NOSQL(NOT only SQL) provides a mechanism for the retrieval and storage
of data.
• At that time, NOSQL did not provide SQL interface but was relational.
• Later when Johan Oskarsson organized an event on open-source
distributed databases, Eric Evans reintroduced the term NOSQL in 2009.
• Most of the NOSQL databases are non-relational, distributed and do not
follow ACID properties.
• The most popular NOSQL databases are MongoDB, Apache Cassandra,
Redis, Hbase, Splunk and Neo4j
Types of NOSQL Data Models
• There are generally four types of NOSQL databases.
• Each of them has their own attributes and limitations.
• You cannot say a single solution is better among all of them because
few databases are meant to solve specific problems.
Comparison of Various NOSQL datamodels
Key-Value Data Model
• Client can get the value for a key, put a value for a key, or delete a key
from the data store.
• The value is a Binary Large Object(BLOB) that is concerned about data
and not what is inside, it is the responsibility of the application to
understand what exactly is stored.
• BLOB is also scalable because it uses primary key access for the
different purposes.
• A sample key/value database is shown in figure
• Examples of popular key/value databases are Riak, Redis, Memcached and
its versions, Berkeley DB, HamsterDB, Amazon DynamoDB, Project
Voldemort and Couchbase
• All key/value databases are not the same, there are major differences
between these products for example Memcached data is not persistent
but Riak data is
• Need to cache user preferences in Memchached datae
• If we implement user preferences in Memcached they would need to be
refreshed entirely in case any of the nodes are lost
• Need to worry about data losing data if stored in Riak database
• Need to consider how to update stale data
• Important to choose the type of key/value database as per requirement
Column-Oriented Data Model
• Based on column
• Every column is considered individually
• Value of a single column stored contiguously
• Example:
• Cassandra
• Big Table
• SimpleDB
• Hbase
• The data maintained by columns are in the form of column-specific files
• The performance on the aggregation queries such as COUNT,SUM,AVG and MAX is high
• Pictorial representation of the column-oriented data model is shown below
Document Data Model
• Many types of Document Data Model like XML,JSON,BSON etc.
• Document databases are
• Self-describing
• Hierarchical tree data structures that contains maps,
collections and scalar values
• Mainly store documents in the value part of the key/value
store
• Value is examinable
• For easier transition from Relational databases, document
databases provide indexing and searching
• Comparison of RDB and DDM is shown in figure below
• Document databases have a different structure to store the
information related to an entity unlike relational database which use
tabular format
• Though the document-oriented model is rich in performance and
scalability ,it doesn’t provide ACID and data integrity that a relational
data model has by definition.