MySQL Day Paris 2018 - MySQL JSON Document Store

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
MySQL Document Store
Copyright © 2018, Oracle and/or its affiliates. All rights reserved.
A Document Store with all the benefits of a Transactional RDBMS
Olivier Dasini
MySQL Principal Solutions Architect EMEA
olivier.dasini@oracle.com
Twitter : @freshdaz
Blog : http://dasini.net/blog

Safe Harbor Statement
The following is intended to outline our general product direction. It is
intended for information purposes only, and may not be incorporated
into any contract. It is not a commitment to deliver any material, code,
or functionality, and should not be relied upon in making purchasing
decisions. The development, release, and timing of any features or
functionality described for Oracle’s products remains at the sole
discretion of Oracle.
2

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 3
Me, Myself & I

MySQL Geek
 Addicted to MySQL for 15+ years!
 Playing with databases for 20+ years

MySQL Writer, Blogger and Speaker
 Also former : DBA, Consultant, Architect, Trainer, ...

MySQL Principal Solutions Architect EMEA at Oracle

Stay tuned! :
 Twitter: @freshdaz
 Blog: http://dasini.net/blog
Olivier DASINI

• Data Integrity
– Normalization
– Constraints (Foreign keys, …)
• Atomicity, Consistency, Isolation, Durability
– ACID Compliant
– Transactions
• SQL
– Powerful Query Language
– Schema, Table, Row
– SELECT / UPDATE / DELETE / INSERT
Relational Databases
5

• An object that can represent structured data
• Structure is implicit in the document; usually no external/central schema
• Compact, popular and standardized
• Can be represented natively in many languages (JavaScript,Python, etc.)
6
• JSON is a lightweight data-interchange format.
• It is easy for humans to read and write.
• It is easy for machines to parse and generate.
• It is based on a subset of the JavaScript Programming.
• Language, Standard ECMA-262 3rd Edition - December 1999.
• Use for storing and exchanging data.
What is a JSON Document ?
JavaScript Object Notation

JSON Support
• Native File Format
– Standardized as ECMA-404 (http://json.org)
– Binary Storage
• Generated Columns
• 20+ Functions
 Search Functions
 Aggregations Functions
• Query structured data and semi-structured JSON data

JSON_ARRAY_APPEND()
JSON_ARRAY_INSERT()
JSON_ARRAY()
JSON_CONTAINS_PATH()
JSON_CONTAINS()
JSON_DEPTH()
JSON_EXTRACT()
JSON_INSERT()
JSON_KEYS()
JSON_LENGTH()
JSON_MERGE[_PRESERVE]()
JSON_OBJECT()
JSON_QUOTE()
JSON_REMOVE()
JSON_REPLACE()
JSON_SEARCH()
JSON_SET()
JSON_TYPE()
JSON_UNQUOTE()
JSON_VALID()
JSON_PRETTY()
JSON_STORAGE_SIZE()
JSON_STORAGE_FREE()
JSON_ARRAYAGG()
JSON_OBJECTAGG()
JSON_MERGE_PATCH()
JSON_TABLE()
JSON Functions
Perform operations on JSON documentsPerform operations on JSON documents
https://dev.mysql.com/doc/refman/8.0/en/json-functions.html

• Schemaless
– No schema design, No normalization, No Foreign Keys, No data types, …
– Very quick initial development
• Flexible data structure
– Embedded arrays or objects
– Valid solution when natural data can’t be modeled optimally into a relational model
– Objects persistence without the use of any ORM – Mapping Object-Oriented
• JSON
– Close to frontend
– Native in JavaScript
– Easy to learn
NoSQL : Document Store
9

How DBAs see data
10
How Devs see data
{
" G N P " : 2 7 3 1 ,
" _ i d " : " M T Q " ,
" N a m e " : " M a r t i n i q u e " ,
" I n d e p Ye a r " : n u l l ,
" g e o g r a p h y " : {
" R e g i o n " : " C a r i b b e a n " ,
" C o n t i n e n t " : " N o r t h A m e r i c a " ,
" S u r f a c e A r e a " : 11 2 8
} ,
" g o v e r n m e n t " : {
" H e a d O f S t a t e " : " E m m a n u e l M a c r o n " ,
" G o v e r n m e n t F o r m " : " O v e r s e a s D e p a r t m e n t o f F r a n c e "
} ,
" d e m o g r a p h i c s " : {
" P o p u l a t i o n " : 3 9 5 0 0 0 ,
" L i f e E x p e c t a n c y " : 7 8 . 3 0 0 0 0 3 0 5 1 7 5 7 8 1
}
}

• Schema (Database)
• Collection
– Group of JSON documents.
– Equivalent of table
• Document
– Set of key-value pairs in JSON format
– Equivalent of row (tuple)
Document Store Databases
11
• CRUD (basic functions of persistent storage)
– CREATE
– READ
– UPDATE
– DELETE
TerminologiesTerminologies

Document Store
An easy, straight forward way to work with JSON documents in MySQL

RDBMS or NoSQL ?
Why not both ?

• One Extensible database
– Do more with the MySQL database
– Many can manage (with deep skills)
– Stable
– Cost-effective
– Easy to move data between like database
types
– Fewer Drivers
– Few Tools
– SQL works, CRUD works
– Operational and Analytical Together
• Many different databases
– Requires larger skill repertoire, more
complex development …
• Harder to find deep skills
– Many Drivers
– Many Tools
– More effort to share and exchange data
– It’s a lot more work
– Operational and Analytical Separate
One Database Many Models VS Many Databases Many Models
14

NoSQL, MySQL, Why not…
• Have both schema-less AND schema in the same technology stack?
• One that checks all the boxes of all stakeholders:
15
Developers:
[✔] Schemaless or/and Schema
[✔] Rapid Prototyping/Simpler APIs
[✔] Document Model
[✔] Transactions
Operations:
[✔] Performance Management/Visibility
[✔] Robust Replication, Backup, Restore
[✔] Comprehensive Tooling Ecosystem
[✔] Simpler application schema upgrades
Business Owner:
[✔] Don’t lose my data = ACID transactions
[✔] Capture all my data = Extensible/Schemaless
[✔] Products On Schedule/Time to Market = Rapid Development

- A One Stop Shop
• Expensive to manage many data stores
• Better few databases – more flexibility
– More developers
• Can harness both NoSQL and SQL savvy
– More DBAs
• No shortage of highly experienced MySQL DBAs
– Less training – don’t need to learn many products
– Cross data store exchange – easier to move from from docs to tables etc.
– One connector/driver needed for apps
16
Combining Relational and Document StoresCombining Relational and Document Stores

Document Store
17

Document Store: Components
• MySQL X PluginMySQL X Plugin
• Introduces X Protocol for relational- and
document operations
• Maps CRUD operations to standard SQL
(relational tables, JSON datatype and
functions)
• X ProtocolX Protocol
• New MySQL client protocol based on top
of industry standard (Protobuf)
• Works for both, CRUD and SQL
operations
• InnoDB ClusterInnoDB Cluster
• Read-Scaling, Write-Scaling, HA
• X DevAPIX DevAPI
• New, modern, async developer API for
CRUD and SQL operations on top of X
Protocol
• Introduces Collections as new Schema obj.
• MySQL ShellMySQL Shell
• Offers interactive X DevAPI mode for app
prototyping
• MySQL ConnectorsMySQL Connectors
• Support for X DevAPI for
• JavaScript, Python, PHP, Java, C#, C++
18

Document Store: Architecture
19

Document Store
SQL is now optional ! - Write applications using X DevAPIX DevAPI
https://insidemysql.com/mysql-8-0-welcome-to-the-devapi/

Welcome to the X DevAPI! - insidemysql.com
MySQL X devAPI connector for:
• Java
– https://insidemysql.com/connector-j-8-0-11-the-face-for-your-brand-new-document-oriented-database/
• .NET
– https://insidemysql.com/introducing-connector-net-with-full-support-for-mysql-8-0/
• Node.JS
– https://insidemysql.com/introducing-connector-node-js-for-mysql-8-0/
• C++
– https://insidemysql.com/what-is-new-in-connector-c-8-0/
• Python
– https://insidemysql.com/using-mysql-connector-python-8-0-with-mysql-8-0/
• PHP
– https://insidemysql.com/introducing-the-mysql-x-devapi-php-extension-for-mysql-8-0/
• ODBC
– https://insidemysql.com/what-is-new-in-connector-odbc-8-0/
21
Motivation
• We are doing something that has not been done before
• Document databases exist! Relational databases exist!
• We even see databases that support relational and document querying over the same data set
• However we have yet to see a relational database include a document model so that a user can use
document objects alongside their existing relational data.

Welcome to the X DevAPI! - PHP Example
22
Writing a program using MySQL 8.0 Document Store
$user = 'my_user';
$passwd = 's0S3kR*T';
$host = 'localhost';
$port = '33060';
$connection_uri = 'mysqlx://'.$user.':'.$passwd.'@'.$host.':'.$port;
$session = mysql_xdevapigetSession($connection_uri);
$schema = $session->getSchema("test");
$collection = $schema->getCollection("my_collection");
$result = $collection->find("Name like :param")->bind(["param" => "Olivier"])->execute();
$docs = $result->fetchAll();
//… print results …
$session->close();
Requirements / Installation
https://dev.mysql.com/doc/apis-php/en/apis-php-mysql-xdevapi.setup.html
https://insidemysql.com/introducing-the-mysql-x-devapi-php-extension-for-mysql-8-0/

Welcome to the X DevAPI! - Python Example
https://insidemysql.com/using-mysql-connector-python-8-0-with-mysql-8-0/
23
Writing a program using MySQL 8.0 Document Store
import mysqlx
session = mysqlx.get_session({
"host": "localhost",
"port": 33060,
"user": "my_user",
"password": "s0S3kR*T"
})
schema = session.get_schema("test")
collection = schema.get_collection("my_collection")
result = collection.find("name like :param").bind("param", "Olivier").limit(1).execute()
docs = result.fetch_all()
print("Name: {0}".format(docs[0]["name"]))
session.close()
Connector/Python 8.0 installation
shell> pip install mysql-connector-python

Welcome to the X DevAPI! - Node.JS Example
24
const session = await mysqlx.getSession({ user: 'root' });
const schema = session.getSchema('docstoretalks');
if (!(await schema.existsInDatabase())) {
await session.createSchema('docstoretalks');
}
await schema.createCollection('sessions', { ReuseExistingObject: true });
const collections = await schema.getCollections();
console.log(collections); // [{ schema: 'docstoretalks', collection: 'sessions' }]
...
$ npm install --save --save-exact @mysql/xdevapi
• For more read “Node.js and the MySQL Document Store” by Rui Quelhas :
– https://www.slideshare.net/rfpquelhas/nodejs-and-the-mysql-document-store
const mysqlx = require('@mysql/xdevapi');

Document Store
Under The Hood

js> session.createSchema('demo')
js> use demo
js> db.getCollections()
js> db.createCollection('myCollection')
js> db.myCollection.add({"param1":"value1", "param2":"value2"})
js> db.myCollection.find()
js> db.myCollection.find().limit(1)
js> db.myCollection.find("_id = '00005af018430000000000000002'")
js> db.myCollection.modify("_id = '1234'").set("param","value")
js> db.myCollection.remove("_id = '1234'")
js> session.startTransaction()
js> …
js> session.rollback()
js> db.myCollection.createIndex("ageIdx",{fields:[{"field":"$.age","type":"INT",required:false}]})
Copyright @ 2018 Oracle and/or its affiliates. All rights reserved.
MySQL Document Store “cheat sheet”
CCreate
RRead
UUpdate
DDelete
Index
TTransaction

Documents and Collections
• Collections are containers for documents
– These documents share a purpose
– Possibly share one or more indexes
– Each collection has a unique name
– Exists within a single schema
• Within a Collection you can
– Add(), Find(), Modify(), and Remove() - JSON documents
• Collections can be
– Create(), List(), Drop()
27

Operation Document Relational
Create Collection.add() Table.insert()
Read Collection.find() Table.select()
Update Collection.modify() Table.update()
Delete Collection.remove() Table.delete()
28
• Use SQL, CRUD APIs – Document (NoSQL) and Relational (SQL), or “All of the Above”
– All of this is in addition to the Classic APIs
Connectors include X DevAPI

Collection Search – find(), bind(), fields()
• Supports many operators to specify searches
– ||, &&, XOR, IS, NOT, BETWEEN, IN, LIKE, !=, <>, >, >=, <, <=, &, |, <<, >>, +, -, *, /, ~, %.
• Searching - find()
– db.CountryInfo.find("GNP > 500000 and demographics.Population < 100000000")
– db.CountryInfo.find("GNP*1000000/demographics.Population > 30000")
• Binding - bind()
– db.CountryInfo.find("Name = :country").bind("country", "Italy")
• Project Results – fields() – returns specific fields
– db.CountryInfo.find("Name = :country").bind("country", "Italy")
29

Setup
30
$ mysqlsh -u user -p --sql
-e"SELECT PLUGIN_NAME, PLUGIN_STATUS, PLUGIN_DESCRIPTION FROM
information_schema.plugins WHERE PLUGIN_NAME='mysqlx'"
+-------------+---------------+--------------------+
| PLUGIN_NAME | PLUGIN_STATUS | PLUGIN_DESCRIPTION |
+-------------+---------------+--------------------+
| mysqlx | ACTIVE | X Plugin for MySQL |
+-------------+---------------+--------------------+
• MySQL 8.0.11 or higher
• X plugin is enabled by default and is part of the MySQL Server
• Check the X plugin installation:
• Disabling at startup:
– Set mysqlx=0 in your MySQL configuration file
– Or by passing in either --mysqlx=0 or --skip-mysqlx when starting the MySQL server

Shell
31
Interface for Development and Administration of MySQLInterface for Development and Administration of MySQL
• Scripting for Javascript, Python, and SQL mode
• Supports MySQL Standard and X Protocols
• Document and Relational Models
• CRUD Document and Relational APIs via scripting
• Traditional Table, JSON, Tab Separated output results formats
• Both Interactive and Batch operations

$ mysqlsh -u <user> -p
…
s
MySQL Shell version 8.0.11
Session type: X
Connection Id: 17
Default schema:
Current schema:
Current user: root@localhost
SSL: Cipher in use: DHE-RSA-AES128-GCM-SHA256 TLSv1.2
Using delimiter: ;
Server version: 8.0.11-commercial MySQL Enterprise Server - Commercial
Protocol version: X protocol
Client library: 8.0.11
Connection: localhost via TCP/IP
TCP port: 33060
Server characterset: utf8mb4
Schema characterset: utf8mb4
Client characterset: utf8mb4
Conn. characterset: utf8mb4
Uptime: 30 min 16.0000 sec
• Following commands provides instructions to begin prototyping database applications
interactively with MySQL Shell for Javascript
• MySQL Shell for Python provide same functionalities

h
===== Global Commands =====
help (?,h) Print this help.
sql Switch to SQL processing mode.
js Switch to JavaScript processing mode.
py Switch to Python processing mode.
source (.) Execute a script file. Takes a file name as an argument.
Start multi-line input when in SQL mode.
quit (q) Quit MySQL Shell.
exit Exit MySQL Shell. Same as quit
connect (c) Connect to a server.
reconnect Reconnect with a server.
option Manage MySQL Shell options.
warnings (W) Show warnings after every statement.
nowarnings (w) Don't show warnings after every statement.
status (s) Print information about the current global connection.
use (u) Set the current schema for the active session.
rehash Update the auto-completion cache with database names.
history View and edit command line history.
For help on a specific command use the command as ? <command>
===== Global Objects =====
dba Enables you to administer InnoDB clusters using the AdminAPI.
mysql Used to work with classic MySQL sessions using SQL.
mysqlx Used to work with X Protocol sessions using the MySQL X DevAPI.
session Represents the currently open MySQL session.
shell Gives access to general purpose functions and properties.
sys Gives access to system specific parameters.
util Global object that groups miscellaneous tools like upgrade checker.
For more help on a global variable use <var>.help(), e.g. dba.help()

MySQL localhost:33060+ ssl JS> session
<Session:root@localhost>
mysql-js> session.help()
...
The following functions are currently supported.
- close Closes the session.
- commit Commits all the operations executed after a call to startTransaction().
- createSchema Creates a schema on the database and returns the corresponding object.
- dropSchema Drops the schema with the specified name.
- getCurrentSchema Retrieves the active schema on the session.
- getDefaultSchema Retrieves the Schema configured as default for the session.
- getSchema Retrieves a Schema object from the current session through it's name.
- getSchemas Retrieves the Schemas available on the session.
- getUri Retrieves the URI for the current session.
- help Provides help about this class and it's members
- isOpen Returns true if session is known to be open.
- quoteName Escapes the passed identifier.
- releaseSavepoint Removes a savepoint defined on a transaction.
- rollback Discards all the operations executed after a call to startTransaction().
- rollbackTo Rolls back the transaction to the named savepoint without terminating the transaction.
- setCurrentSchema Sets the current schema for this session, and returns the schema object for it.
- setFetchWarnings Enables or disables warning generation.
- setSavepoint Creates or replaces a transaction savepoint with the given name.
- sql Creates a SqlExecute object to allow running the received SQL statement on the target
MySQL Server.
- startTransaction Starts a transaction context on the server.

CRUD operations
Collection creation
35

js> session.createSchema('doc')
<Schema:doc>
js> use doc
Default schema `doc` accessible through db.
js> db
<Schema:doc>
[]
js> db.createCollection('posts')
<Collection:posts>
[
<Collection:posts>
]
js> db.posts.add({"title":"MySQL 8.0 rocks", "text":"My first post!", "code": "42"})
Query OK, 1 item affected (0.0221 sec)
js> db.posts.add({"title":"Polyglot database", "text":"Developing both SQL and NoSQL applications"})

CRUD operations
Find documents
37
https://dev.mysql.com/doc/refman/8.0/en/mysql-shell-tutorial-javascript-documents-find.html

js> db.posts.add("This is not a valid JSON document")
CollectionAdd.add: Argument #1 expected to be a document, JSON expression or a list of documents (ArgumentError)
js> db.posts.find()
[
{
"_id": "00005aec27830000000000000001",
"code": "42",
"text": "My first post!",
"title": "MySQL 8.0 rocks"
},
{
"_id": "00005aec27830000000000000002",
"text": "Developing both SQL and NoSQL applications",
"title": "Polyglot database"
}
]
js> db.posts.find().fields("text")
[
{
"text": "My first post!"
},
{
"text": "Developing both SQL and NoSQL applications"
}
]

js> db.posts.find().limit(1)
[
{
"_id": "00005aec27830000000000000001",
"code": "42",
}
]
js> db.posts.add({"_id": "00005aec27830000000000000001", "code": "13"})
ERROR: 5116: Document contains a field value that is not unique but required to be
js> db.posts.find().limit(1).skip(1)
[
{
"_id": "00005aec27830000000000000002",
}
]

js> db.posts.find("title = 'Polyglot database'")
[
{
"_id": "00005aec27830000000000000002",
}
]
js> db.posts.find("title = :what").bind("what", "Polyglot database")
[
{
"_id": "00005aec27830000000000000002",
}
]

js> db.posts.find()
[
{
"_id": "00005aec27830000000000000001",
"code": "42",
},
{
"_id": "00005aec27830000000000000002",
}
]
js> db.posts.find().sort("text") /* same as : db.posts.find().sort("text asc")*/
[
{
"_id": "00005aec27830000000000000002",
},
{
"_id": "00005aec27830000000000000001",
"code": "42",
}
]

CRUD operations
Document modification
42
https://dev.mysql.com/doc/refman/8.0/en/mysql-shell-tutorial-javascript-documents-modify.html

js> db.posts.find("_id = '00005aec27830000000000000002'")
[
{
"_id": "00005aec27830000000000000002",
}
]
js> db.posts.modify("_id = '00005aec27830000000000000002'").set("title", "Hybrid database")
js> db.posts.find("_id = '00005aec27830000000000000002'").fields("title", "text")
[
{
"title": "Hybrid database"
}
]
js> db.posts.modify("_id = '00005aec27830000000000000002'").set("author" ,"Daz").set("code" , 2018)
js> db.posts.find("_id = '00005aec27830000000000000002'").fields("author","text","code")
[
{
"author": "Daz",
"code": 2018,
"text": "Developing both SQL and NoSQL applications"
}
]

js> db.posts.modify("_id = '00005aec27830000000000000002'").set("autor", "Daz le magnifique")
[
{
"_id": "00005aec27830000000000000002",
"author": "Daz",
"autor": "Daz le magnifique",
"code": 2018,
}
]
js> db.posts.modify("_id = '00005aec27830000000000000002'").unset("autor")
[
{
"_id": "00005aec27830000000000000002",
"author": "Daz",
"code": 2018,
}
]

CRUD operations
Full ACID
45
https://dev.mysql.com/doc/x-devapi-userguide/en/transaction-handling.html

It relies on the proven MySQL InnoDB’s strength & robustness:
• innodb_flush_log_at_trx_commit = 1
• innodb_doublewrite = ON
• sync_binlog = 1
• transaction_isolation = REPEATABLE-READ | READ-COMMITTED | ...
We do care about your data!
Document Store is Full ACID
ACID transactions = Don’t lose my dataACID transactions = Don’t lose my data
46

Document Store supports transactions
ACID transactions = Don’t lose my dataACID transactions = Don’t lose my data
js> session.startTransactionstartTransaction() // Start a transaction
Query OK, 0 rows affected (0.0004 sec)
js> db.posts.modify("_id ='00005aec27830000000000000002'").set("title","ACID Transaction").set("text","Don’t lose my data")
js> db.posts.find("_id ='00005aec27830000000000000002'").fields(["title", "text"])
[
{
"text": "ACID Transaction",
"title": "Don’t lose my data"
}
]
js> db.restaurants.remove("_id='5ad5b645f88c5bb8fe3fd337'")
js> db.restaurants.find("_id='5ad5b645f88c5bb8fe3fd337'")
Empty set (0.0005 sec)
js> session.rollback() // Rollback the transaction (if necessary e.g. in case of an error)
Query OK, 0 rows affected (0.0862 sec)
js> db.posts.find("_id ='00005aec27830000000000000002'").fields(["title", "text"])
[
{
… [snip] …
js> db.restaurants.find("_id='5ad5b645f88c5bb8fe3fd337'")
[
{
… [snip] …
47

CRUD operations
Document deletion
48
https://dev.mysql.com/doc/refman/8.0/en/mysql-shell-tutorial-javascript-documents-remove.html

js> db.posts.add({"actions": {"type": 1, "site": "http://dasini.net/blog/fr"}})
js> db.posts.add({"actions": {"type": 2, "site": "http://dasini.net/blog/en"}})
js> db.posts.find().sort("actions.type desc").limit(1)
[
{
"_id": "00005aec2783000000000000000b",
"actions": {
"site": "http://dasini.net/blog/en",
"type": 2
}
}
]
js> db.posts.remove("true").sort("actions.type desc").limit(1)
js> db.posts.find('actions.type IN (1, 2)')
[
{
"_id": "00005aec2783000000000000000a",
"actions": {
"site": "http://dasini.net/blog",
"type": 1
}
}
]
js> db.posts.remove('actions.type = 1')
js> db.posts.find('actions.type IN (1, 2)')
Empty set (0.0005 sec)

CRUD operations
Indexes
50
https://dev.mysql.com/doc/refman/8.0/en/mysql-shell-tutorial-javascript-indexes-create.html

// Add a Non Unique Index
js> db.posts.createIndex("idxAuthor", {fields: [{field: "$.author", type: "TEXT(20)"}]})
// Add a Unique Index
js> db.posts.createIndex("idxCode", {fields: [{field: "$.code", type: "INT"}], unique: true})
js> db.posts.add({code: 42})
ERROR: 5116: Document contains a field value that is not unique but required to be
// required:true => field is required to exist in the document i.e. NOT NULL
js> db.posts.createIndex("idxTitle", {fields: [{field: "$.title", type: "TEXT(30)", required:
true}]})
js> db.posts.add({code: 11})
ERROR: 5115: Document is missing a required field
// Drop an Index
js> db.posts.dropIndex("idxTitle")
https://dev.mysql.com/doc/x-devapi-userguide/en/collection-indexing.html

Using SQL with Session
52

// pics is a regular SQL table that contains JSON documents
js> session.sql("SELECT * FROM pics")
+----+------+---------------------+----------------------------------------------------------------+
| id | code | ts | doc |
+----+------+---------------------+----------------------------------------------------------------+
| 1 | 42 | 2018-04-25 14:56:19 | {"data": {"tags": [1, 11], "type": "image", "location": "Fr"}} |
| 2 | 2018 | 2018-04-25 14:47:15 | {"data": {"tags": [2], "type": "image", "location": "Fr"}} |
+----+------+---------------------+----------------------------------------------------------------+
// SQL join between a collection posts and a table pics
js> session.sql("SELECT po.doc->'$.title' title, po.doc->'$.text' msg, ts time, pi.doc details FROM
posts po INNER JOIN pics pi ON (pi.code = po.doc->>'$.code') WHERE pi.code = 2018")
+-------------------+----------------------------------------------+---------------------+------------------------------------------------------------+
| title | msg | time | details |
+-------------------+----------------------------------------------+---------------------+------------------------------------------------------------+
| "Hybrid database" | "Developing both SQL and NoSQL applications" | 2018-04-25 14:47:15 | {"data": {"tags": [2], "type": "image", "location": "Fr"}} |
+-------------------+----------------------------------------------+---------------------+------------------------------------------------------------+
// Many other SQL commands available
js> session.sql("CREATE …
js> session.sql("CALL...
js> session.sql("DROP...
Using SQL with Session
• The Session object has a sql() function that takes any SQL statement as a string
• Possible to join collections with tables :-O

CRUD operations – SQL / Relational
54

CRUD operations – SQL / Relational Tables
55
• You can use MySQL Shell to manipulate not just JSON documents, but also relational tables.
• Basic operations scoped by tables include:
Operation Form Description
db.name.insert()
The insert() method inserts one or more records into the
named table.
db.name.select()
The select() method returns some or all records in the
named table.
db.name.update() The update() method updates records in the named table.
db.name.delete()
The delete() method deletes one or more records from the
named table.

js> db.getTables()
[
<Table:pics>
]
// INSERT Records into table pics
js> db.pics.insert('code', 'doc').values(42, '{"data": {"tags": [1, 11], "type": "image", "location": "Fr"}}')
js> db.pics.insert('code', 'doc').values(11, '{"data": {"tags": [2], "type": "image", "location": "Fr"}}')
js> db.pics.insert('code', 'doc').values(11, '{"data": {"tags": [9, 99], "type": "sound"}}')
// Select table pics
js> db.pics.select()
+----+--------+---------------------+----------------------------------------------------------------+
+----+--------+---------------------+----------------------------------------------------------------+
| 3 | 11 | 2018-04-25 12:12:12 | {"data": {"tags": [9, 99], "type": "sound"}} |
+----+--------+---------------------+----------------------------------------------------------------+
js> db.pics.select(["id", "ts", "doc"])
+----+---------------------+----------------------------------------------------------------+
| id | ts | doc |
+----+---------------------+----------------------------------------------------------------+
| 1 | 2018-04-25 12:11:33 | {"data": {"tags": [1, 11], "type": "image", "location": "Fr"}} |
| 2 | 2018-04-25 12:12:01 | {"data": {"tags": [2], "type": "image", "location": "Fr"}} |
| 3 | 2018-04-25 12:12:12 | {"data": {"tags": [9, 99], "type": "sound"}} |
+----+---------------------+----------------------------------------------------------------+
js> db.pics.select(["ts", "doc"]).where("id = 1")
+---------------------+----------------------------------------------------------------+
| ts | doc |
+---------------------+----------------------------------------------------------------+
| 2018-04-25 12:11:33 | {"data": {"tags": [1, 11], "type": "image", "location": "Fr"}} |
+---------------------+----------------------------------------------------------------+

js> db.pics.select(["ts", "doc"]).where("id like :id").bind("id", 1)
+---------------------+----------------------------------------------------------------+
| ts | doc |
+---------------------+----------------------------------------------------------------+
+---------------------+----------------------------------------------------------------+
js> db.pics.select(["ts", "doc"]).where('id IN (1, 2) AND ts < NOW()').orderBy(["ts desc"]).limit(2)
+---------------------+----------------------------------------------------------------+
| ts | doc |
+---------------------+----------------------------------------------------------------+
| 2018-04-25 12:12:01 | {"data": {"tags": [2], "type": "image", "location": "Fr"}} |
+---------------------+----------------------------------------------------------------+
js> db.pics.select().where("doc->'$.data.type' = 'image'")
+----+--------+---------------------+----------------------------------------------------------------+
+----+--------+---------------------+----------------------------------------------------------------+
+----+--------+---------------------+----------------------------------------------------------------+
js> db.pics.select().where("doc->'$.data.tags[0]' = 9")
+----+------+---------------------+----------------------------------------------+
+----+------+---------------------+----------------------------------------------+
+----+------+---------------------+----------------------------------------------+
// Is pics a view?
js> db.pics.isView()
False

+----+--------+---------------------+----------------------------------------------------------------+
+----+--------+---------------------+----------------------------------------------------------------+
+----+--------+---------------------+----------------------------------------------------------------+
// UPDATE Table pics
js> db.pics.update().set('code', 2018).where("doc->'$.data.type' = 'sound'")
+----+------+---------------------+----------------------------------------------------------------+
+----+------+---------------------+----------------------------------------------------------------+
+----+------+---------------------+----------------------------------------------------------------+
// DELETE records from table pics
js> db.pics.delete().where("doc->'$.data.tags[0]' = 2")
+----+------+---------------------+----------------------------------------------------------------+
+----+------+---------------------+----------------------------------------------------------------+
+----+------+---------------------+----------------------------------------------------------------+
js> //db.pics.delete() // Drop all rows

Relational / SQL / Analytics on MySQL Tables
59

SQL for running Complex and/or analytic queries
• Relational Analytics
– Collations are MySQL tables
– So you can use ALL the power of MySQL’s SQL commands i.e.
●
NATURAL [ INNER | CROSS | {LEFT|RIGHT} OUTER ] JOIN
●
CTE
●
Window functions
●
JSON_ARRAYAGG() / JSON_OBJECTAGG() / JSON_PRETTY()
●
…

// Calculate the average of the restaurant rating
sql>
SELECT rname AS Restaurant, Style, avg(rating) AS 'Average Rating'
FROM restaurants, JSON_TABLE(doc, "$" columns( rname varchar(50) path "$.name",
style varchar(50) path "$.cuisine", nested path "$.grades[*]" columns (rating int
path "$.score" ))) as jt
GROUP BY rname, style
ORDER BY avg(rating) DESC;
+--------------------------------+------------+----------------+
| Restaurant | Style | Average Rating |
+--------------------------------+------------+----------------+
| Wendy'S | Hamburgers | 13.7500 |
| Dj Reynolds Pub And Restaurant | Irish | 9.2500 |
| Morris Park Bake Shop | Bakery | 8.2000 |
+--------------------------------+------------+----------------+
• Simple query with JSON_TABLE
– JSON_TABLE: Extracts data from a JSON document and returns it as a relational table having the specified columns

// Calculate the average of the restaurant rating (more elegant form)
sql>
SELECT doc->>"$.name" AS Restaurant, doc->>"$.cuisine" AS Cuisine,
(SELECT AVG(score) FROM JSON_TABLE(doc, "$.grades[*]" COLUMNS (score INT PATH
"$.score")) AS r) AS 'Avg Rating'
FROM restaurants
ORDER BY 'Avg Rating' DESC;
+--------------------------------+------------+------------+
| Restaurant | Cuisine | Avg Rating |
+--------------------------------+------------+------------+
| Morris Park Bake Shop | Bakery | 8.2000 |
| Wendy'S | Hamburgers | 13.7500 |
| Dj Reynolds Pub And Restaurant | Irish | 9.2500 |
+--------------------------------+------------+------------+
• Simple query with JSON_TABLE
– JSON_TABLE: Extracts data from a JSON document and returns it as a relational table having the specified columns

// Calculate the average of the restaurant rating (Adding CTE and window functions)
sql>
WITH cte1 AS (
SELECT doc->>"$.name" AS Restaurant, doc->>"$.cuisine" AS Cuisine,
(SELECT AVG(score) FROM JSON_TABLE(doc, "$.grades[*]" COLUMNS (score INT PATH
"$.score")) AS r) AS AvgScore FROM restaurants
)
SELECT *, RANK() OVER (PARTITION BY cuisine ORDER BY AvgScore) AS `Rank`
FROM cte1
ORDER BY `Rank`, AvgScore DESC;
+--------------------------------+------------+----------+------+
| Restaurant | Cuisine | AvgScore | Rank |
+--------------------------------+------------+----------+------+
| Wendy'S | Hamburgers | 13.7500 | 1 |
| Dj Reynolds Pub And Restaurant | Irish | 9.2500 | 1 |
| Morris Park Bake Shop | Bakery | 8.2000 | 1 |
+--------------------------------+------------+----------+------+
• Commont Table Expression (CTE) : https://dev.mysql.com/doc/refman/8.0/en/with.html
• Windows functions : https://dev.mysql.com/doc/refman/8.0/en/window-functions.html

Document Store
Wrap-up

Document Store
✔ Built on Proven SQL/InnoDB/Replication technology
✔ Schema-less/Relational/Hybrid
✔ ACID/Transactions
✔ No SQL required - CRUD/JSON/Documents/Indexes
✔ Modern Dev API
✔ Modern/Efficient Protocol
✔ SQL Queries/Analytics over JSON Documents
✔ Transparent and Easy HA/Scaling with MySQL InnoDB Cluster
65

Hybrid database that covers 4 core areas
Relational
MySQL Tables and SQL
Relational
Relational Analytics
Relational Analytics
Document Store
MySQL Collections and
Table with JSON
Datatype
Key/Value
MySQL Memcached
Document Store
MySQL Collections and
Table with JSON
Datatype
Key/Value
MySQL Memcached
Mass Scale Analytics
Sharded MySQL
Relational and/or
Document Store
Mass Scale Analytics
Sharded MySQL
Relational and/or
Document Store
66
Operational Analytical
Relational/SQL
NoSQL

Clients and Applications
InnoDB Storage Engine
mysqld process
Firewall
Audit
Encryption
Authentication
Online Backup
Monitoring
Integration
Support
3rd
Party
Tools3rd
Party
Tools3rd
Party
Tools3rd
Party
Tools3rd
Party
Tools
NoSQLNoSQL
Simple access
patterns
Compromise on consistency
for performance
Ad-hoc data format
Simple operation
SQLSQL
Complex queries with
joins
ACID transactions
Well defined schemas
Rich set of tools
Document Store
Works transparently with MySQL Enterprise FeaturesWorks transparently with MySQL Enterprise Features
Masking

Resources
Topic Link(s)
Top 10 reasons for NoSQL with MySQL http://lefred.be/content/top-10-reasons-for-nosql-with-mysql/
MySQL as a Document Store https://dev.mysql.com/doc/refman/8.0/en/document-store.html
MySQL Shell User Guide https://dev.mysql.com/doc/mysql-shell-excerpt/8.0/en/index.html
MySQL Shell Documentation https://dev.mysql.com/doc/dev/mysqlsh-api-javascript/8.0/
https://dev.mysql.com/doc/dev/mysqlsh-api-python/8.0/
X Dev API User Guide https://dev.mysql.com/doc/x-devapi-userguide/en/
X Plugin https://dev.mysql.com/doc/refman/8.0/en/x-plugin.html
MySQL Engineering Blog https://insidemysql.com/mysql-8-0-welcome-to-the-devapi/
https://insidemysql.com/mysql-document-store-crud-quick-start/
MySQL JSON Data Type https://dev.mysql.com/doc/refman/8.0/en/json.html
Blogs http://dasini.net/blog/2018/07/23/30-mins-with-mysql-json-functions/
http://dasini.net/blog/2015/11/17/30-mins-avec-json-en-mysql/
http://dasini.net/blog/2015/11/30/json-et-colonnes-generees-avec-mysql
http://mysqlserverteam.com/tag/json/
http://mysqlserverteam.com/category/docstore/

Thank you!

MySQL Day Paris 2018 - MySQL JSON Document Store

MySQL Day Paris 2018 - MySQL JSON Document Store

More Related Content

What's hot (20)

Similar to MySQL Day Paris 2018 - MySQL JSON Document Store (20)

More from Olivier DASINI (11)

Recently uploaded (20)

MySQL Day Paris 2018 - MySQL JSON Document Store