BIG DATA & ANALYTICS

UNIT III NOSQL

Brief History of NoSQL Databases

•1998- Carlo Strozzi use the term NoSQL for his lightweight, open-source relational
database
•2000- Graph database Neo4j is launched
•2004- Google BigTable is launched
•2005- CouchDB is launched
•2007- The research paper on Amazon Dynamo is released
•2008- Facebooks open sources the Cassandra project
•2009- The term NoSQL was reintroduced
What is NoSQL?

Carl Strozz introduced the NoSQL concept in 1998

NoSQL is a non-relational DBMS, that does not require a fixed schema, avoids joins,
and is easy to scale.

NoSQL database is used for distributed data stores with humongous data storage
needs. NoSQL is used for Big data and real-time web apps.

Used by Twitter, Facebook, Google - Collect terabytes of user data.

NoSQL database stands for "Not Only SQL" or "Not SQL." Though a better term would
NoREL NoSQL caught on.
NoSQL database system encompasses a wide range of database technologies

NoSQL store structured, semi-structured, unstructured and polymorphic data.

Why NoSQL?
To deal with huge volumes of data, RDBMS response time becomes slow.
To overcome this we need to "scale up" our systems by upgrading our existing
hardware, which is expensive.

Solution: Distribute database, add hosts whenever the load increases. This method is
known as "scaling out.
Features of NoSQL
1. Non-relational
2. Schema-free
3. Simple API
4. Distributed

1. Non-relational
• NoSQL databases never follow the relational model
• Never provide tables with flat fixed-column records
• Work with self-contained aggregates or BLOBs - Binary Large OBject (BLOB)
• Doesn't require object-relational mapping and data normalization
• No complex features like query languages, query planners, referential integrity joins,
ACID
2. Schema-free

• NoSQL databases are either schema-free or have relaxed schemas

• Do not require any sort of definition of the schema of the data
• Offers heterogeneous structures of data in the same domain
3. Simple API

• Offers easy to use interfaces for storage and querying data provided

• APIs allow low-level data manipulation & selection methods

• Text-based protocols mostly used with HTTP REST with JSON

• Mostly used no standard based query language

• Web-enabled databases running as internet-facing services

4. Distributed

Multiple NoSQL databases can be executed in a distributed fashion

•Offers auto-scaling and fail-over capabilities
•Often ACID concept can be sacrificed for scalability and throughput
•Mostly no synchronous replication between distributed nodes Asynchronous Multi-
Master replication, peer-to-peer, HDFS Replication
•Only providing eventual consistency
•Shared Nothing Architecture. This enables less coordination and higher distribution.
Types of NoSQL Databases

Four categories of NoSQL databases. Each of these categories has its unique attributes
and limitations. No specific database is better to solve all problems. You should select a
database based on your product needs.
1. Key-value Pair Based
2. Document-oriented
3. Column-oriented
4. Graphs based
1. Key Value Pair Based

Data is stored in key/value pairs.

It is designed to handle lots of data and heavy load.

Key-value pair storage databases store data as a hash table where each key is
unique, and the value can be a JSON, BLOB(Binary Large Objects), string, etc.

For example, key like "Website" associated with a value like

"www.pbsiddhartha.ac.in".
Key Value Pair is most basic types of NoSQL databases.
Used as a collection, dictionaries, associative arrays, etc.
Key value stores help the developer to store schema-less data.
Work best for shopping cart contents.

e.g: Redis, Dynamo, Riak. They are all based on Amazon's Dynamo paper.
2. Document-Oriented:

Stores and retrieves data as a key value pair but the value part is stored as a
document.
The document is stored in JSON or XML formats.
The value is understood by the DB and can be queried.

Relational Vs Document
A document database, you have data store like JSON object. You do not require to
define which make it flexible.

Used for CMS systems, blogging platforms, real-time analytics & e-commerce
applications.

Not Suitable for complex transactions which require multiple operations or queries
against varying aggregate structures.

e.g - Amazon SimpleDB, CouchDB, MongoDB, Riak, Lotus Notes

3. Column-based

Column-oriented databases work on columns and are based on BigTable paper by Google.
Every column is treated separately.
Values of single column databases are stored contiguously.
They deliver high performance on aggregation queries like SUM, COUNT, AVG, MIN
etc. as the data is readily available in a column.

Column-based NoSQL databases are widely used to manage data warehouses,

business intelligence, CRM, Library card catalogs,

e.g. - HBase, Cassandra, Hypertable

4.Graph-Based

A graph type database stores entities as well the relations amongst those entities.

The entity is stored as a node with the relationship as edges.

An edge gives a relationship between nodes.

Every node and edge has a unique identifier.

A Graph database is a multi-relational in nature.

Traversing relationship is fast as they are already captured into the DB, and there is
no need to calculate them.

Graph base database mostly used for social networks, logistics, spatial data.

e.g - Neo4J, Infinite Graph, OrientDB, FlockDB

Query Mechanism tools for NoSQL

The most common data retrieval mechanism is the REST-based retrieval of a value based
on its key/ID with GET resource

* REST stands for REpresentational State Transfer.

Document store Database offers more difficult queries as they understand the value in a
key-value pair.

For example, CouchDB allows defining views with MapReduce

Advantages of NoSQL

• Can be used as Primary or Analytic Data Source

• Big Data Capability

• No Single Point of Failure

• Easy Replication

• No Need for Separate Caching Layer

• It provides fast performance and horizontal scalability.

• Can handle structured, semi-structured, and unstructured data with equal effect

• Object-oriented programming which is easy to use and flexible

• NoSQL databases don't need a dedicated high-performance server

• Support Key Developer Languages and Platforms

• Simple to implement than using RDBMS

• It can serve as the primary data source for online applications.

• Handles big data which manages data velocity, variety, volume, and complexity

• Excels at distributed database and multi-data center operations

• Eliminates the need for a specific caching layer to store data

•Offers a flexible schema design which can easily be altered without downtime or
service disruption
Disadvantages of NoSQL

• No standardization rules

• Limited query capabilities

• RDBMS databases and tools are comparatively mature
• It does not offer any traditional database capabilities, like consistency when
multiple transactions are performed simultaneously.
• When the volume of data increases it is difficult to maintain unique values as
keys become difficult
• Doesn't work as well with relational data
• The learning curve is stiff for new developers
• Open source options so not so popular for enterprises.