PHP, The X DevAPI, and the MySQL Document Store Presented January 23rd, 2019 at Benelux PHP Conference

PHP,
The X DevAPI,
and the
MySQL Document Store
Dave Stokes
@Stoker
David.Stokes@Oracle.com
https://elephantdolphin.blogspot.com/
https://slideshare.net/davidmstokes

Safe Harbor Agreement
THE FOLLOWING IS INTENDED TO OUTLINE OUR GENERAL PRODUCT
DIRECTION. IT IS INTENDED FOR INFORMATION PURPOSES ONLY, AND
MAY NOT BE INCORPORATED INTO ANY CONTRACT. IT IS NOT A
COMMITMENT TO DELIVER ANY MATERIAL, CODE, OR FUNCTIONALITY,
AND SHOULD NOT BE RELIED UPON IN MAKING PURCHASING DECISIONS.
THE DEVELOPMENT, RELEASE, AND TIMING OF ANY FEATURES OR
FUNCTIONALITY DESCRIBED FOR ORACLE'S PRODUCTS REMAINS AT THE
SOLE DISCRETION OF ORACLE.
2

Programmers Tasks
● 20 years ago
○ Editor - vi, vim, emacs
○ Compiler - 1 or two
languages
○ Debugger
○ Browser (maybe)
○ XML (maybe)
○ HTML (growing need)
○ Source Control System
○ IDE (you wish!)
○ Documentation (low priority)
4

Programmers Tasks
● 20 years ago
languages
○ Debugger
○ Browser (maybe)
○ XML (maybe)
○ IDE (you wish!)
● Today
○ GIT
○ Tests
○ CI
○ Small(er) stuff
■ Containers, VMs, micro
services, etc.
○ Main language &
Framework
○ 7 or 8 JavaScript
Frameworks
○ Data Store (SQL & NoSQL) 5

Programmers Tasks
● 20 years ago
languages
○ Debugger
○ Browser (maybe)
○ XML (maybe)
○ IDE (you wish!)
● Today
○ GIT
○ Tests
○ CI
○ Small(er) stuff
■ Containers, VMs, micro services,
etc.
○ Main language & Framework
○ 7 or 8 JavaScript Frameworks
○ Data Store (SQL & NoSQL)
○ JSON
○ SSH/TLS
○ Encryption
○ Bash & Powershell
○ Markdown
○ Cloud
6

Programmers Tasks
● 20 years ago
languages
○ Debugger
○ Browser (maybe)
○ XML (maybe)
○ IDE (you wish!)
● Today
○ GIT
○ Tests
○ CI
○ Small(er) stuff
■ Containers, VMs, micro services, etc.
○ JSON
○ SSH/TLS
○ Encryption
○ Markdown
○ Cloud
○ Debugger
○ Multiple Browsers
○ Multiple IDEs
○ Documentation (still low priority)
○ Mentoring (giving or receiving)
○ More frameworks
○ More third party libraries
○ Embedded, Android, IOS, etc.
○ Key management
○ Slack
○ Repository tools
○ * as a Service (*aaS)
○ Whatever the latest craze is from The Register, Inforworld, Slashdot
7

Programmers Tasks
● 20 years ago
languages
○ Debugger
○ Browser (maybe)
○ XML (maybe)
○ IDE (you wish!)
● Today
○ GIT
○ Tests
○ CI
○ Small(er) stuff
■ Containers, VMs, micro services, etc.
○ JSON
○ SSH/TLS
○ Encryption
○ Markdown
○ Cloud
○ Debugger
○ Multiple Browsers
○ Multiple IDEs
○ Documentation (still low priority)
○ Mentoring (giving or receiving)
○ More frameworks
○ More third party libraries
○ Embedded, Android, IOS, etc.
○ Key management
○ Slack
○ Repository tools
○ * as a Service (*aaS)
○ Whatever the latest craze is from The Register, Inforworld, Slashdot
○ YAML and other markup file syntax
○ The newest Google tool
○ etc
8

9
BTW This was the big concern of 1999!

Constantly Increasing Learning Curve
10

2%
In my not so scientific study over the past eight years using informal informational
gathering techniques, roughly TWO PERCENT of developers receive any formal
training in Structured Query Language (SQL), Relational Calculus, Data
Normalization, or other database technologies.
And 100% of that 2% wonder why their queries stink! 11

Declarative Language Buried in a OO/Procedural
A big problem for many
developers is that they
are used to Object
Oriented and/or
Procedural programming
languages.
SQL is declarative
programming language
and embedding SQL in
PHP is an object-
relational impedance
mismatch.
// Formulate Query
// This is the best way to perform an SQL query
// For more examples, see mysql_real_escape_string()
$query = sprintf("SELECT firstname, lastname, address, age FROM
friends
WHERE firstname='%s' AND lastname='%s'",
mysql_real_escape_string($firstname),
mysql_real_escape_string($lastname));
// Perform Query
$result = mysql_query($query);
http://us1.php.net/manual/en/function.mysql-query.php
13
Note: the ‘mysql’ connector has
been deprecated and is not
supported in PHP 7 and was
replaced by ‘mysqli’ but there is still
a lot of code with ‘mysql’ around

Other Issues
1. UGLY
2. Hard to have help from your IDE
3. Extra level of complexity / opportunity to fail
4. Badly organized data & queries

And Yet More Issues
● Relational tables need to be set up
● Indexes
● Data mutability
● Need to rely on a DBA (or someone who has that role)
● Can’t start coding much of project before data format is know
● ORMs -- more complexity and another layer to maintain
15

X DevAPI &
The MySQL Document Store
16

So what if there was a way to ...
● Use schemaless JSON documents so you do not have to normalize data and
code before you know the complete schema
● Not have to embed SQL strings in your code
● Use a modern programming style API
● Be able to use the JSON data from SQL or NoSQL -
○ Best of both worlds
17

The X DevAPI wraps powerful concepts in a simple API.
A new high-level session concept enables you to write code that can transparently scale from single MySQL
Server to a multiple server environment.
→Read operations are simple and easy to understand.
→Non-blocking, asynchronous calls follow common host language patterns.
→The X DevAPI introduces a new, modern and easy-to-learn way to work with your data.
Documents are stored in Collections and have their dedicated CRUD operation set.
→Work with your existing domain objects or generate code based on structure definitions for
strictly typed languages.
→Focus is put on working with data via CRUD operations.
→Modern practices and syntax styles are used to get away from traditional SQL-String-Building.
19

Scale from Single Server to Cluster w/o Code Change
The code that is needed to connect to a MySQL document store looks a lot like
the traditional MySQL connection code, but now applications can establish logical
sessions to MySQL server instances running the X Plugin. Sessions are produced
by the mysqlx factory, and the returned Sessions can encapsulate access to one
or more MySQL server instances running X Plugin. Applications that use Session
objects by default can be deployed on both single server setups and database
clusters with no code changes.
20

var mysqlx = require('mysqlx');
// Connect to server on localhost
var mySession = mysqlx.getSession( {
host: 'localhost', port: 33060,
user: 'user', password: 'password' } );
var myDb = mySession.getSchema('test');
// Use the collection 'my_collection'
var myColl = myDb.getCollection('my_collection');
// Specify which document to find with Collection.find() and
// fetch it from the database with .execute()
var myDocs = myColl.find('name like :param').limit(1).
bind('param', 'S%').execute();
// Print document
print(myDocs.fetchOne());
mySession.close();
21

PECL MySQl X DevAPI
http://php.net/manual/en/book.mysql-xdevapi.php
22

Installation (Ubuntu 18.04 & PHP 7.2)
Note: The MySQL X DevAPI PECL extension is not bundled with PHP.
// Dependencies
$ apt install build-essential libprotobuf-dev libboost-dev openssl protobuf-
compiler
// PHP with the desired extensions; php7.2-dev is required to compile
$ apt install php7.2-cli php7.2-dev php7.2-mysql php7.2-pdo php7.2-xml
// Compile the extension
$ pecl install mysql_xdevapi
// Use the 'phpenmod' command
$ phpenmod -v 7.2 -s ALL mysql_xdevapi 23

A JAON document is a data structure composed of
key-value pairs and is the fundamental structure for using
MySQL as document store.
This document shows that the values of keys can be
simple data types, such as integers or strings, but can also
contain other documents, arrays, and lists of documents.
For example, the geography key's value consists of
multiple key-value pairs. A JSON document is represented
internally using the MySQL binary JSON object, through
the JSON MySQL datatype.
The most important differences between a document and
the tables known from traditional relational databases are
that the structure of a document does not have to be
defined in advance, and a collection can contain multiple
documents with different structures. Relational tables on
the other hand require that their structure be defined, and
all rows in the table must contain the same columns.
{
"GNP": .6,
"IndepYear": 1967,
"Name": "Sealand",
"_id": "SEA",
"demographics": {
"LifeExpectancy": 79,
"Population": 27
},
"geography": {
"Continent": "Europe",
"Region": "British Islands",
"SurfaceArea": 193
},
"government": {
"GovernmentForm": "Monarchy",
"HeadOfState": "Michael Bates"
}
}
24

#!/bin/php
<?php
$session = mysql_xdevapigetSession("mysqlx://myuser:mypass@localhost:33060");
if ($session === NULL) {
die("Connection could not be established");
}
$marco = [
"name" => "Marco",
"age" => 19,
"job" => "Programmer"
];
$mike = [
"name" => "Mike",
"age" => 39,
"job" => "Manager"
];
$schema = $session->getSchema("testxx");
$collection = $schema->createCollection("example1");
$collection = $schema->getCollection("example1");
$collection->add($marco, $mike)->execute();
var_dump($collection->find("name = 'Mike'")->execute()->fetchOne()); 25
An Example

#!/bin/php
<?php
}
$marco = [
"name" => "Marco",
"age" => 19,
];
$mike = [
"name" => "Mike",
"age" => 39,
"job" => "Manager"
];
The URI specifies the details of the connection
- Protocol
- Username
- Authentication String
- Lost
- Port
The X Plugin listens at port 33060

#!/bin/php
<?php
}
$marco = [
"name" => "Marco",
"age" => 19,
];
$mike = [
"name" => "Mike",
"age" => 39,
"job" => "Manager"
];
1. Connect to the schema ‘testxx’
2. Create a collection for JSON documents
named ‘example1’
3. Use the collection ‘example1’

#!/bin/php
<?php
}
$marco = [
"name" => "Marco",
"age" => 19,
];
$mike = [
"name" => "Mike",
"age" => 39,
"job" => "Manager"
];
Add records to the collection

#!/bin/php
<?php
}
$marco = [
"name" => "Marco",
"age" => 19,
];
$mike = [
"name" => "Mike",
"age" => 39,
"job" => "Manager"
];
Find the record where the name = ‘Mike’

The Emphasis is on CRUD
30
Operation form Description
db.name.add()
The add() method inserts one document or more documents into the named
collection.
db.name.find() The find() method returns some or all documents in the named collection.
db.name.modify() The modify() method updates documents in the named collection.
db.name.remove()
The remove() method deletes one document or a list of documents from the
named collection.
CRUD EBNF Definitions - https://dev.mysql.com/doc/x-devapi-userguide/en/mysql-x-
crud-ebnf-definitions.html

No more messy strings
$SQLQuery = “SELECT * FROM people WHERE job
LIKE “ . $job . “AND age > $age”;
Versus
$collection = $schema-
>getCollection("people");
$result = $collection
->find('job like :job and age > :age')
->bind(['job' => 'Butler', 'age' => 16])
->execute(); 32

Easier to read/comprehend than SQL
$result = $collection
->remove('age > :age_from and age < :age_to')
->bind(['age_from' => 20, 'age_to' => 50])
->limit(2)
->execute();
33
Easy to add filters like SORT, LIMIT, HAVING GROUP BY

Indexes on collections
$collection->createIndex(
'myindex1',
'{"fields": [{
"field": "$.name",
"type": "TEXT(25)",
"required": true}],
"unique": false}'
);
34

Got Tables?
You can also use the MySQL Document Store with Relational Tables
35

Quick Example using a table
#!/bin/php
<?php
$session = mysql_xdevapigetSession("mysqlx://root:hidave@localhost:33060");
}
$schema = $session->getSchema("world");
$table = $schema->getTable("city");
$row = $table->select('Name','District')
->where('District like :district')
->bind(['district' => 'Texas'])
->limit(25)
->execute()->fetchAll();
print_r($row);
36

The URI Connection for a Session
#!/bin/php
<?php
}
->limit(25)
$row = $result->fetchAll();
print_r($row);
37

Get schema and table
#!/bin/php
<?php
}
->limit(25)
print_r($row);
38

The Query
#!/bin/php
<?php
}
->limit(25)
;
print_r($row);
39

What if I Don’t Want to Rewrite old queries?
$session->sql("CREATE DATABASE addressbook")->execute();
40

Okay I can use Collections and Tables
BUT WHAT
ABOUT
BOTH!?!?!?! 41

You can also use a Collection as a table!
$collection = $schema->getCollection("people");
$table = $schema->getCollectionAsTable("people");
42

#!/bin/php
<?php
}
$schema = $session->getSchema("nyeats");
$table = $schema->getTable("restaurants");
$sqlx =
"WITH cte1 AS (SELECT doc->>"$.name" AS 'name',
doc->>"$.cuisine" AS 'cuisine',
(SELECT AVG(score) FROM
JSON_TABLE(doc, "$.grades[*]"
COLUMNS (score INT PATH "$.score")) as r ) AS avg_score
FROM restaurants)
SELECT *, rank() OVER
(PARTITION BY cuisine ORDER BY avg_score) AS `rank`
FROM cte1
ORDER by `rank`, avg_score DESC limit 10";
$row->sql($sqlx)->execute()-fetchAll();
print_r($row);
43
You can also use
Windowing Functions
for Advanced
Analytics

One of the advantages
of using MySQL as a
NoSQL Document
store is that you can
use SQL analytics on
your data!
44

Combine CTEs, Windowing Functions, & JSON_TABLE
WITH cte1 AS (SELECT doc->>"$.name"
AS 'name',
doc->>"$.cuisine" AS 'cuisine',
(SELECT AVG(score) FROM
JSON_TABLE(doc, "$.grades[*]"
COLUMNS (score INT PATH
"$.score")) as r ) AS avg_score
FROM restaurants)
SELECT *, rank() OVER
(PARTITION BY cuisine ORDER BY
avg_score) AS `rank`
FROM cte1
ORDER by `rank`, avg_score DESC limit 10
JSON_TABLE turns unstructured
JSON documents in to temporary
relational tables that can be
processed with SQL
Windowing Function for analytics
Common Table Expression make it
easy to write sub-queries
45

The X Plugin
… is a shared object that is installed by default in MySQL 8.0 and must be loaded
in 5.7
mysqlsh -u user -h localhost --classic --dba enableXProtocol
Or
mysql> INSTALL PLUGIN mysqlx SONAME 'mysqlx.so';
It listens on port 33060 so make sure you open firewall for 3306 (old MySQL) and
33060 (X Plugin).
And it supports SSL/TLS!!!
47

The New MySQL Shell
48
Built In JavaScript and
Python interpreters let
you work with you data
in the MySQL Shell.
Plus you get command
completion, great help
facilities, the ability to
check for server
upgrades, and the
ability to administrate a
InnoDB Clusters.
And you can also use
SQL

50
InnoDB Cluster
MySQL InnoDB cluster provides a complete high availability
solution for MySQL. Each MySQL server instance runs MySQL
Group Replication, which provides the mechanism to replicate
data within InnoDB clusters, with built-in failover.
AdminAPI removes the need to work directly with Group
Replication in InnoDB clusters MySQL Router can automatically
configure itself based on the cluster you deploy, connecting client
applications transparently to the server instances.
Multiple secondary server instances are replicas of the primary. If
the primary fails, a secondary is automatically promoted to the
role of primary. MySQL Router detects this and forwards client
applications to the new primary. Advanced users can also
configure a cluster to have multiple-primaries.

Questions and Answers plus Additional Resources
● More Info on MySQL Document Store
○ PHP PECL Extension for X DevAPI
■ http://php.net/manual/en/book.mysql-xdevapi.php
○ MySQL Document Store
■ https://dev.mysql.com/doc/refman/8.0/en/document-store.html
○ X DevAPI User Guide
■ https://dev.mysql.com/doc/x-devapi-userguide/en/
○ Dev.MySQL.com for Downloads and Other Doces
○ X DevAPI Tutorial for Sunshine PHP
■ https://github.com/davidmstokes/PHP-X-DevAPI
● David.Stokes@Oracle.com
○ https://elephantdolphin.blogspot.com/
○ Slides at https://slideshare.net/davidmstokes
○ @Stoker
○ MySQL & JSON - A Practical Programming Guide
51

PHP, The X DevAPI, and the MySQL Document Store Presented January 23rd, 2019 at Benelux PHP Conference

Recommended

More Related Content

What's hot (20)

Similar to PHP, The X DevAPI, and the MySQL Document Store Presented January 23rd, 2019 at Benelux PHP Conference (20)

More from Dave Stokes (18)

Recently uploaded (20)

PHP, The X DevAPI, and the MySQL Document Store Presented January 23rd, 2019 at Benelux PHP Conference