Big

Data Exercises
Fall 2016 Week 11 ETH Zurich

Introduction
This exercise will cover document stores. As a representative of document stores, MongoDB was chosen for the practical
exercises. You can install it on your local machine or use Azure Portal.

1. Document stores
A record in document store is a document. Document encoding schemes include XML, YAML, JSON, and BSON, as well as binary
forms like PDF and Microsoft Office documents (MS Word, Excel, and so on). MongoDB documents are similar to JSON objects.
Documents are composed of field-value pairs and have the following structure:

The values of fields may include other documents, arrays, and arrays of documents. Data in MongoDB has a flexible schema in
the same collection. All documents do not need to have the same set of fields or structure, and common fields in a collection's
documents may hold different types of data.

Questions
1. What are advantages of document stores over relational databases?
2. Can the data in document stores be normalized?
3. How does denormalization affect performance?
4. How does a large number of small documents affect performance?
5. What makes document stores different from key-value stores?

2. Data models in document stores

Create a mapping between the following data models and MongoDB documents. Data models may be chosen twice.

1. One-to-one relationship with embedded documents

2. One-to-one relationship with normalized data model
3. One-to-many relationship with embedded documents
4. One-to-many relationship with normilized documents(at least in 1NF)
5. One-to-many relationship with document references
6. Tree structure with parent references
7. Tree structure with children references
8. Tree structure with an array of ancestors
9. Tree structure with materialized paths
10. Tree structure with nested sets
a.

{
name: "O'Reilly Media",
founded: 1980,
location: "CA",
books: [12346789, 234567890, ...]
}

{
_id: 123456789,
title: "MongoDB: The Definitive Guide",
author: [ "Kristina Chodorow", "Mike Dirolf" ],
published_date: ISODate("2010-09-24"),
pages: 216,
language: "English"
}

{
_id: 234567890,
title: "50 Tips and Tricks for MongoDB Developer",
author: "Kristina Chodorow",
published_date: ISODate("2011-05-06"),
pages: 68,
language: "English"
}
b.

{ _id: "MongoDB", parent: "Databases" }

{ _id: "dbm", parent: "Databases" }
{ _id: "Databases", parent: "Programming" }
{ _id: "Languages", parent: "Programming" }
{ _id: "Programming", parent: "Books" }
{ _id: "Books", parent: null }

c.

{
_id: "joe",
name: "Joe Bookreader"
}

{
patron_id: "joe",
street: "123 Fake Street",
city: "Faketon",
state: "MA",
zip: "12345"
}

d.

{ _id: "MongoDB", children: [] }

{ _id: "dbm", children: [] }
{ _id: "Databases", children: [ "MongoDB", "dbm" ] }
{ _id: "Languages", children: [] }
{ _id: "Programming", children: [ "Databases", "Languages" ] }
{ _id: "Books", children: [ "Programming" ] }

e.

{
_id: "joe",
name: "Joe Bookreader"
}

{
patron_id: "joe",
street: "123 Fake Street",
city: "Faketon",
state: "MA",
zip: "12345"
}

{
patron_id: "joe",
street: "1 Some Other Street",
city: "Boston",
state: "MA",
zip: "12345"
}
f.

{
_id: "joe",
name: "Joe Bookreader",
address: {
street: "123 Fake Street",
city: "Faketon",
state: "MA",
zip: "12345"
}
}

g.

{ _id: "Books", path: null }

{ _id: "Programming", path: ",Books," }
{ _id: "Databases", path: ",Books,Programming," }
{ _id: "Languages", path: ",Books,Programming," }
{ _id: "MongoDB", path: ",Books,Programming,Databases," }
{ _id: "dbm", path: ",Books,Programming,Databases," }

h.

{
_id: "oreilly",
name: "O'Reilly Media",
founded: 1980,
location: "CA"
}

{
_id: 123456789,
title: "MongoDB: The Definitive Guide",
author: [ "Kristina Chodorow", "Mike Dirolf" ],
published_date: ISODate("2010-09-24"),
pages: 216,
language: "English",
publisher_id: "oreilly"
}

{
_id: 234567890,
title: "50 Tips and Tricks for MongoDB Developer",
author: "Kristina Chodorow",
published_date: ISODate("2011-05-06"),
pages: 68,
language: "English",
publisher_id: "oreilly"
}
i.

{ _id: "MongoDB", ancestors:["Books","Programming","Databases"], parent:"Databases"}

{ _id: "dbm", ancestors: ["Books", "Programming", "Databases"], parent: "Databases"}
{ _id: "Databases", ancestors: ["Books", "Programming"], parent: "Programming"}
{ _id: "Languages", ancestors: ["Books", "Programming"], parent: "Programming"}
{ _id: "Programming", ancestors: ["Books" ], parent: "Books"}
{ _id: "Books", ancestors: [ ], parent: null }

j.

{
_id: "joe",
name: "Joe Bookreader",
addresses: [
{
street: "123 Fake Street",
city: "Faketon",
state: "MA",
zip: "12345"
},
{
street: "1 Some Other Street",
city: "Boston",
state: "MA",
zip: "12345"
}
]
}

k.

{ _id: "Books", parent: 0, left: 1, right: 12 }

{ _id: "Programming", parent: "Books", left: 2, right: 11 }
{ _id: "Languages", parent: "Programming", left: 3, right: 4 }
{ _id: "Databases", parent: "Programming", left: 5, right: 10 }
{ _id: "MongoDB", parent: "Databases", left: 6, right: 7 }
{ _id: "dbm", parent: "Databases", left: 8, right: 9 }
3. MongoDB

3.1 Install MongoDB

MongoDB is an open-source document database. The next step is to install it on your local machine. For that, you can follow
general instruction on MongoDB web page install MongoDB (https://docs.mongodb.com/manual/administration/install-
community/)

Here the short instruction for Ubuntu is outlined:

1. Import the public key used by the package management system.

sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv EA312927

2. Create the /etc/apt/sources.list.d/mongodb-org-3.2.list list file using the command appropriate for your version of Ubuntu:

Ubuntu 12.04

echo "deb http://repo.mongodb.org/apt/ubuntu precise/mongodb-org/3.2 multiverse" | sudo tee /et

c/apt/sources.list.d/mongodb-org-3.2.list

Ubuntu 14.04

echo "deb http://repo.mongodb.org/apt/ubuntu trusty/mongodb-org/3.2 multiverse" | sudo tee /etc

/apt/sources.list.d/mongodb-org-3.2.list

Ubuntu 16.04

echo "deb http://repo.mongodb.org/apt/ubuntu xenial/mongodb-org/3.2 multiverse" | sudo tee /etc

/apt/sources.list.d/mongodb-org-3.2.list

3. Install the MongoDB packages.

sudo apt-get update

sudo apt-get install -y mongodb-org

To run MongoDB, execute the following command:

sudo service mongod start

To stop MongoDB, execute the following command:

sudo service mongod stop

3.2 Import the dataset
Retrieve the "restaurants" dataset from

https://raw.githubusercontent.com/mongodb/docs-assets/primer-dataset/primer-dataset.json
(https://raw.githubusercontent.com/mongodb/docs-assets/primer-dataset/primer-dataset.json)

Use mongoimport to insert the documents into the restaurants collection in the test database. If the collection already exists
in the test database, the operation will drop the restaurants collection first.

mongoimport --db test --collection restaurants --drop --file ./primer-dataset.json

3.3 Mongo shell

The mongo shell is an interactive JavaScript interface to MongoDB. You can use the mongo shell to query and update data as well
as to perform administrative operations.

To start mongo use:

`
mongo --shell

In the mongo shell connected to a running MongoDB instance, switch to the test database.

test

Try to insert a document into the restaurants collection. In addition, you can see the structure of documents the in the
collection.

db.restaurants.insert(
{
"address" : {
"street" : "2 Avenue",
"zipcode" : "10075",
"building" : "1480",
"coord" : [ -73.9557413, 40.7720266 ]
},
"borough" : "Manhattan",
"cuisine" : "Italian",
"grades" : [
{
"date" : ISODate("2014-10-01T00:00:00Z"),
"grade" : "A",
"score" : 11
},
{
"date" : ISODate("2014-01-16T00:00:00Z"),
"grade" : "A",
"score" : 17
}
],
"name" : "Vella",
"restaurant_id" : "41704620"
}
)

Query all documents in a collection:

db.restaurants.find()

Query one document in a collection:

db.restaurants.findOne()

To format the printed result, you can add .pretty() to the operation, as in the following:

db.restaurants.find().limit(1).pretty()
Query Documents
For the db.collection.find() method, you can specify the following optional fields:

a query filter to specify which documents to return,

a query projection to specifies which fields from the matching documents to return (the projection limits the amount of
data that MongoDB returns to the client over the network),
optionally, a cursor modifier to impose limits, skips, and sort orders.

3.4 Questions
Write queries that return the following:

1. All restaurants in borough (a town) "Brooklyn" and cuisine (a style of cooking) "Hamburgers".
2. The number of restaurants in the borough "Brooklyn" and cuisine "Hamburgers".
3. All restaurants with zipcode 11225.
4. Names of restaurants with zipcode 11225 that have at least one grade "C".
5. Names of restaurants with zipcode 11225 that have as first grade "C" and as second grade "A".
6. Names and streets of restaurants that don't have an "A" grade.
7. All restaurants with a grade C and a score greater than 50.
8. All restaurants with a grade C or a score greater than 50.
9. All restaurants that have only A grades.
10. A table with zipcode and number of restaurants that are in the borough "Queens" and have "Brazilian" cuisine.

You can read more about it here:

https://docs.mongodb.com/getting-started/shell/query/ (https://docs.mongodb.com/getting-started/shell/query/)

https://docs.mongodb.com/getting-started/shell/aggregation/ (https://docs.mongodb.com/getting-started/shell/aggregation/)

https://docs.mongodb.com/manual/aggregation/ (https://docs.mongodb.com/manual/aggregation/)

4. Indexing in MongoDB
Indexes support the efficient resolution of queries. Without indexes, MongoDB must scan every document of a collection to select
those documents that match the query statement. Scan can be highly inefficient and require MongoDB to process a large volume
of data.

Indexes are special data structures that store a small portion of the data set in an easy-to-traverse form. The index stores the
value of a specific field or set of fields, ordered by the value of the field as specified in the index.

MongoDB supports indexes that contain either a single field or multiple fields depending on the operations that this index type
supports.

By default, MongoDB creates the _id index, which is an ascending unique index on the _id field, for all collections when the
collection is created. You cannot remove the index on the _id field.
Managing indexes in MongoDB
An explain operator provides information on the query plan. It returns a document that describes the process and indexes used
to return the query. This may provide useful insight when attempting to optimize a query.

db.restaurants.find({"borough" : "Brooklyn").explain()

In the mongo shell, you can create an index by calling the createIndex() method.

db.restaurants.createIndex( { borough : 1 })

Now, you retrieve a new query plan for indexed data.

db.restaurants.find({"borough" : "Brooklyn").explain()

The value of the field in the index specification describes the kind of index for that field. For example, a value of 1 specifies an
index that orders items in ascending order. A value of -1 specifies an index that orders items in descending order.

To remove all indexes, you can use db.collection.dropIndexes(). To remove a specific index you can use
db.collection.dropIndex(), such as db.restaurants.dropIndex({ borough : 1 }).

Questions
1. Write a query that can benefit from the following index:

db.restaurants.createIndex( { borough : 1 })

2. We have an index on address field as follows:

db.restaurants.createIndex( { address : -1 })

Will the query

db.restaurants.find({"address.zipcode" : "11225" })

use that index?

3. Write a command for creating an index on the zipcode field.
4. Let us have the compound index:

db.restaurants.createIndex( { "borough": 1, "cuisine": -1 } )

Which of the following queries use the index above?

A. db.restaurants.find({"borough" : "Brooklyn")
B. db.restaurants.find({"cuisine" : "Hamburgers")
C. db.restaurants.find({"borough" : "Brooklyn", "cuisine" : "Hamburgers" })
D. db.restaurants.find().sort( {borough: -1} )
E. db.restaurants.find().sort( {borough: 1, cuisine: 1 } )
F. db.restaurants.find().sort( {borough: -1, cuisine: 1 } )
G. db.restaurants.find().sort( {cuisine: 1, borough: -1 } )
5. Answer Question 4, but for the following index:

db.restaurants.createIndex( { "borough": 1, "cuisine": -1, "name" : -1} )

6. Is it possible to create the index below? Why?/Why not?

db.restaurants.createIndex( { "address.coord": 1, "grades": -1} )

7. Write an index to speed up the following query:

db.restaurants.find({"grades.grade" : { $ne : "A"} } , {"name" : 1 , "address.street": 1})

8. Write an index to speed up the following query:

db.restaurants.find({"grades.score" : {$gt : 50} , "grades.grade" : "C"})

9. What are the differences between two index strategies below

A. db.restaurants.createIndex( { "borough": 1, "cuisine": -1} )
B. db.restaurants.createIndex( { "borough": 1} )
db.restaurants.createIndex( { "cuisine": -1} )
10. How are sparse indexes different from normal ones?
5. SQL to MongoDB Mapping
Create a one-to-one mapping between the following SQL and MongoDB queries.

1. INSERT INTO users(user_id, age, status)

VALUES ("bcd001", 45, "A")

2. SELECT * FROM users

3. SELECT user_id, status FROM users

4. SELECT * FROM users

WHERE age > 25 AND age <= 50

5. SELECT * FROM users

WHERE status = "A" OR age = 50

6. CREATE TABLE users (

id MEDIUMINT NOT NULL AUTO_INCREMENT,
user_id Varchar(30),
age Number,
status char(1),
PRIMARY KEY (id)
)

7. SELECT COUNT(user_id)
FROM users

a.

db.users.find(
{ age: { $gt: 25, $lte: 50 } }
)

b.

db.users.find(
{ },
{ user_id: 1, status: 1, _id: 0 }
)

c.

db.createCollection("users")

d.

db.users.insert(
{ user_id: "bcd001", age: 45, status: "A" }
)

e.

db.users.count( { user_id: { $exists: true}})

f.

db.users.find(
{ $or: [ { status: "A" } ,
{ age: 50 } ] }
)

g.

db.users.find()
6. True or False
Say if the following statements are true or false.

1. In document stores, you must determine and declare a table's schema before inserting data.
2. Documents stores are not subject to data modeling and support only one denormalized data model.
3. Different relationships between data can be represented by references and embedded documents.
4. MongoDB provides the capability to validate documents during updates and insertions.
5. There are no joins in MongoDB.