SlideShare a Scribd company logo

Practical JSON in MySQL 5.7 and Beyond

Ike Walker
Ike Walker

The document discusses the evolution and practical use of JSON in MySQL from 1995 to the present, highlighting its integration as a data type and the associated parsing capabilities. It details various use cases for JSON, such as flexible rollups, configuration data, and addressing the entity-attribute-value (EAV) model while avoiding common pitfalls. The document also illustrates practical examples of how to store, retrieve, and manipulate JSON data within MySQL databases.

Technology
1 of 71
Downloaded 29 times
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
How people build software ! " Practical JSON in MySQL 5.7 and Beyond IkeWalker GitHub Percona Live April 27, 2017
How people build software! What this talk is about 2 !
How people build software! ! What this talk is about 3 • Using JSON in MySQL
How people build software! What this talk is not about 4 !
How people build software! ! What this talk is not about 5 • Whether you should use JSON in MySQL YES! NO!
How people build software! A brief history of JSON in MySQL 6 !
How people build software! ! Timeline: MySQL and JSON 7 1995: MYSQL 2002: JSON 2011: COMMON_SCHEMA 2013: MYSQL JSON UDFS 2015: NATIVE JSON
How people build software! 2002 - 2011 “There’s an app for that” 8 !
How people build software! ! 2002-2011: “There’s an app for that” 9 • Store JSON as text • Rewrite full string every time • Parsing happens exclusively in the application layer • Or write your own stored procedures/functions BUILD JSON WRITE TO DB STORE AS TEXT READ FROM DB PARSE JSON
How people build software! 2011 - 2013 “Standard procedures” 10 !
How people build software! ! 2011-2013: “Standard procedures” 11 • Store JSON as text • Rewrite full string every time • Some simple parsing can be done with common_schema: • get_option() • extract_json_value()
How people build software! ! 2011-2013: “Standard procedures” 12 mysql> select common_schema.extract_json_value(f.event_data,'/age') as age, -> common_schema.extract_json_value(f.event_data,'/gender') as gender, -> sum(f.event_count) as event_count -> from json_event_fact f -> group by age, gender; +----------+---------+-------------+ | age | gender | event_count | +----------+---------+-------------+ | Over 30 | female | 3710983 | | Over 30 | male | 2869302 | | Under 30 | female | 5027591 | | Under 30 | male | 4918382 | | unknown | female | 42039 | | unknown | male | 50173 | | unknown | unknown | 8372 | +----------+---------+-------------+
How people build software! 2013 - 2015 “Lab experiments” 13 !
How people build software! ! 2013-2015: “Lab experiments” 14 • Store JSON as text • Some updates can be done with UDFs • Much faster parsing supported by UDFs
How people build software! ! 2013-2015: “Lab experiments” 15 mysql> select json_extract(f.event_data,'state') as state, -> json_extract(f.event_data,'age') as age, -> json_extract(f.event_data,'gender') as gender, -> sum(f.event_count) as event_count -> from json_event_fact f -> group by state, age, gender; +-------+----------+--------+-------------+ | state | age | gender | event_count | +-------+----------+--------+-------------+ | CA | 18-24 | female | 5384 | | CA | 18-24 | male | 791 | | CA | 18-24 | null | 25 | | CA | 25-34 | female | 6731 | | CA | 25-34 | male | 1292 | | CA | 25-34 | null | 38 | | CA | 35-44 | female | 10638 | | CA | 35-44 | male | 1822 | | CA | 35-44 | null | 20 | +-------+----------+--------+-------------+
How people build software! 2015 - present “Going native” 16 !
How people build software! ! 2015-present: “Going native” 17 • Store JSON as text or JSON datatype • JSON datatype is binary, with keys sorted • Fast access to embedded data • Updates executed via native functions • Extensive parsing supported by native functions
How people build software! JSON function examples 18 !
How people build software! ! JSON_EXTRACT 19 mysql> select json_unquote(json_extract(event_data,'$.country’)) as country, -> sum(event_count) as events -> from json_event_fact -> where d = current_date() - interval 1 day -> and ad_id = 2 -> group by country; +---------+--------+ | country | events | +---------+--------+ | nl | 107954 | | us | 27373 | +---------+--------+ 2 rows in set (0.00 sec)
How people build software! ! JSON_SEARCH 20 mysql> select json_search(event_data,'one','android') as json_path -> from json_event_fact -> having json_path is not null -> limit 1; +-----------+ | json_path | +-----------+ | "$.os" | +-----------+ 1 row in set (0.01 sec) mysql> select json_search('{"fa":"la","la":["la","la"]}','all','la')G *************************** 1. row *************************** json_search('{"fa":"la","la":["la","la"]}','all','la'): ["$.fa", "$.la[0]", "$.la[1]"] 1 row in set (0.00 sec)
How people build software! ! JSON_REPLACE 21 mysql> set @json = cast('{"foo":"bar"}' as json); Query OK, 0 rows affected (0.00 sec) mysql> set @json = json_replace(@json, '$.foo', 'UPDATED'); Query OK, 0 rows affected (0.00 sec) mysql> select @json; +--------------------+ | @json | +--------------------+ | {"foo": "UPDATED"} | +--------------------+ 1 row in set (0.00 sec)
How people build software! ! JSON_ARRAY 22 mysql> set @json = cast('{"foo":"bar"}' as json); Query OK, 0 rows affected (0.00 sec) mysql> set @json = json_replace(@json, '$.foo', json_array('bar', 'car', 'far')); Query OK, 0 rows affected (0.01 sec) mysql> select @json; +--------------------------------+ | @json | +--------------------------------+ | {"foo": ["bar", "car", "far"]} | +--------------------------------+ 1 row in set (0.00 sec)
How people build software! ! JSON_OBJECT 23 mysql> set @json = json_object( -> 'id','007', -> 'name','James Bond', -> 'cars',json_array('Alfa Romeo','Aston Martin','BMW') -> ); Query OK, 0 rows affected (0.00 sec) mysql> select @jsonG *************************** 1. row *************************** @json: {"id": "007", "cars": ["Alfa Romeo", "Aston Martin", "BMW"], "name": "James Bond"} 1 row in set (0.00 sec)
How people build software! ! JSON COMPARATOR 24 mysql> select cast('"hello"' as json) = 'hello'; 1 mysql> select cast(42 as json) = 42; 1 mysql> select cast(false as json) = false; 1 mysql> select cast('{"foo":"bar"}' as json) = cast('{"foo":"bar"}' as json); 1
How people build software! ! But wait! There’s more! 25 https://dev.mysql.com/doc/refman/5.7/en/json-function-reference.html
How people build software! Use case #1 Flexible rollups 26 !
How people build software! ! Flexible rollups 27 • GOAL: Support different fields for multiple customers within a single dimension
How people build software! ! Flexible rollups 28 For example, some ads track age and gender and others track country and os: {"age":"Over 30","gender":"female"} {"country":"us","os":"android"}
How people build software! ! Flexible rollups 29 -- try to create rollup with JSON datatype mysql> create table json_event_fact ( -> d date not null, -> ad_id int not null, -> event_data json not null, -> event_count int not null, -> primary key (d,ad_id,event_data) -> ); ERROR 3152 (42000): JSON column 'event_data' cannot be used in key specification.
How people build software! ! Flexible rollups 30 -- use text instead mysql> create table json_event_fact ( -> d date not null, -> ad_id int not null, -> event_data varchar(750) not null, -> event_count int not null, -> primary key (d,ad_id,event_data) -> ); Query OK, 0 rows affected (0.05 sec)
How people build software! ! Flexible rollups 31 -- generated column hack mysql> create table json_event_fact ( -> d date not null, -> ad_id int not null, -> event_data json not null, -> event_data_text varchar(750) as (cast(event_data as char)) stored, -> event_count int not null, -> primary key (d,ad_id,event_data_text) -> ); Query OK, 0 rows affected (0.16 sec)
How people build software! ! Flexible rollups 32 mysql> select event_data->'$.age' as age, -> sum(event_count) as events -> from json_event_fact -> where d = current_date() - interval 1 day -> and ad_id = 1 -> group by age; +------------+---------+ | age | events | +------------+---------+ | "Over 30" | 810424 | | "Under 30" | 1205544 | +------------+---------+ 2 rows in set (0.03 sec)
How people build software! ! Flexible rollups 33 mysql> select json_unquote(event_data->'$.country') as country, -> sum(event_count) as events -> from json_event_fact -> where d = current_date() - interval 1 day -> and ad_id = 2 -> group by country; +---------+--------+ | country | events | +---------+--------+ | nl | 107954 | | us | 27373 | +---------+--------+ 2 rows in set (0.00 sec)
How people build software! Use case #2 Configuration data 34 !
How people build software! ! Configuration data 35 • GOAL: Store extensible configuration data of arbitrary depth for each ad
How people build software! ! Configuration data 36 create table ad_config_data ( ad_config_data_id int not null primary key, ad_id int not null, name varchar(50) not null, config_data json not null );
How people build software! ! Configuration data 37 { "a56a81eb": { "type": "AD", "uuid": "d9e9d8ae-8a33-11e6-97e0-22000b93579c", "subConfig": {}, "dataSupport": true }, "a6529578": { "type": "VIDEO", "uuid": "e09b40af-8a33-11e6-97e0-22000b93579c", "subConfig": { "video_url": "https://www.youtube.com/watch?v=dQw4w9WgXcQ", "hd": "true” } }, "a6caab6e": { "type": "AD", "uuid": "e89a3877-8a33-11e6-97e0-22000b93579c", "subConfig": {} } }
How people build software! ! Configuration data 38 { "paths": { "path_3007ea93": ["action_312c40f4"], "path_30972a80": ["action_3158f2a7", "action_3185a6da", "action_31aedd9b"], … }, "actions": { "action_312c40f4": { … } } }
How people build software! ! Configuration data 39 mysql> select json_extract(config_data,'$.paths.path_30c06190') as sub_paths from ad_config_data where ad_id = 1 and name = 'actions'G *************************** 1. row *************************** sub_paths: ["action_5b5343af", "action_5b8b6c39", "action_5bbb05d6"] 1 row in set (0.00 sec) mysql> update ad_config_data -> set config_data = json_replace(config_data,'$.paths.path_30c06190',json_array('action_5b5343af', 'action_5bbb05d6')) -> where ad_id = 1 and name = 'actions'; Query OK, 1 row affected (0.00 sec) Rows matched: 1 Changed: 1 Warnings: 0 mysql> select json_extract(config_data,'$.paths.path_30c06190') as sub_paths from ad_config_data where ad_id = 1 and name = 'actions'G *************************** 1. row *************************** sub_paths: ["action_5b5343af", "action_5bbb05d6"] 1 row in set (0.00 sec)
How people build software! ! Configuration data 40 mysql> select json_extract(config_data,'$.*.type') -> from ad_config_data -> where ad_id = 1 -> and name = 'layers'; +--------------------------------------+ | json_extract(config_data,'$.*.type') | +--------------------------------------+ | ["AD", "VIDEO", "AD"] | +--------------------------------------+ 1 row in set (0.00 sec)
How people build software! ! Configuration data 41 mysql> select json_search(config_data,'one','action_312c40f4') -> from ad_config_data -> where ad_id = 1 -> and name = 'actions'; +--------------------------------------------------+ | json_search(config_data,'one','action_312c40f4') | +--------------------------------------------------+ | "$.paths.path_3007ea93[0]" | +--------------------------------------------------+ 1 row in set (0.00 sec)
How people build software! Use case #3 EAV antidote 42 !
How people build software! ! EAV antidote 43 • GOAL: Store Entity-Attribute-Value style data while avoiding the EAV antipattern
How people build software! ! EAV antidote 44 • The EAV antipattern is described well by Bill Karwin in his book: “SQL Antipatterns” • In Bill’s example the EAV anti-pattern is used to store two types of issues (bugs and features) in a single shared table.
How people build software! ! EAV antidote 45 CREATE TABLE Issues ( issue_id SERIAL PRIMARY KEY ); CREATE TABLE IssueAttributes ( issue_id BIGINT UNSIGNED NOT NULL, attr_name VARCHAR(100) NOT NULL, attr_value VARCHAR(100), PRIMARY KEY (issue_id, attr_name), FOREIGN KEY (issue_id) REFERENCES Issues(issue_id) ); ATTRIBUTES STORED AS KEY-VALUE PAIRS
How people build software! ! EAV antidote 46 CREATE TABLE Issues ( issue_id SERIAL PRIMARY KEY, reported_by BIGINT UNSIGNED NOT NULL, product_id BIGINT UNSIGNED, priority VARCHAR(20), version_resolved VARCHAR(20), status VARCHAR(20), issue_type VARCHAR(10), -- BUG or FEATURE attributes TEXT NOT NULL, -- all dynamic attributes for the row FOREIGN KEY (reported_by) REFERENCES Accounts(account_id), FOREIGN KEY (product_id) REFERENCES Products(product_id) );
How people build software! ! EAV antidote 47 CREATE TABLE Issues ( issue_id SERIAL PRIMARY KEY, reported_by BIGINT UNSIGNED NOT NULL, product_id BIGINT UNSIGNED, priority VARCHAR(20), version_resolved VARCHAR(20), status VARCHAR(20), issue_type VARCHAR(10), -- BUG or FEATURE attributes JSON NOT NULL, -- all dynamic attributes for the row severity VARCHAR(20) AS (attributes->"$.severity"), -- only for bugs version_affected VARCHAR(20) AS (attributes->"$.version_affected"), -- only for bugs sponsor VARCHAR(50) AS (attributes->"$.sponsor"), -- only for feature requests FOREIGN KEY (reported_by) REFERENCES Accounts(account_id), FOREIGN KEY (product_id) REFERENCES Products(product_id) );
How people build software! JSON + Generated Columns Two great techs that tech great together. 48 !
How people build software! ! JSON + Generated Columns 49 • Allows you to expose one or more JSON fields as table columns • Supports indexes • Choose virtual (not stored) unless the index is Primary Key, FULLTEXT, or GIS
How people build software! ! Example #1: Add age column to rollup table 50 mysql> alter table json_event_fact -> add column age varchar(20) as (event_data->'$.age'); Query OK, 0 rows affected (0.19 sec) Records: 0 Duplicates: 0 Warnings: 0 mysql> select age, -> sum(event_count) as events -> from json_event_fact -> where d = current_date() - interval 1 day -> and ad_id = 1 -> group by age; +------------+---------+ | age | events | +------------+---------+ | "Over 30" | 810424 | | "Under 30" | 1205544 | +------------+---------+ 2 rows in set (0.00 sec)
How people build software! ! Example #2: Add gender column/index to rollup 51 mysql> alter table json_event_fact -> add column gender varchar(20) as (event_data->'$.gender'), -> add index (gender); Query OK, 0 rows affected (0.08 sec) Records: 0 Duplicates: 0 Warnings: 0 mysql> select sum(event_count) as events -> from json_event_fact -> where gender = '"female"'; +---------+ | events | +---------+ | 1081286 | +---------+ 1 row in set (0.00 sec)
How people build software! JSON vs TEXT JSON as text versus JSON datatype 52 !
How people build software! ! TEXT vs. JSON datatype 53 • Use the JSON datatype in general • There are a few exceptions. Use text types if you need to: • Include the column in a primary key • Store heterogeneous data (some rows are strings, some are JSON) • Store JSON with depth greater than 100
How people build software! ! TEXT vs. JSON datatype 54 JSON data type sorts by keys: mysql> select cast('{"c":"3","b": {"2":"2","1":"1"},"a":"1"}' as json) as key_sort_test; +-------------------------------------------------+ | key_sort_test | +-------------------------------------------------+ | {"a": "1", "b": {"1": "1", "2": "2"}, "c": "3"} | +-------------------------------------------------+ 1 row in set (0.00 sec)
How people build software! ! TEXT vs. JSON datatype 55 • Use the JSON_VALID() function to test validity of existing column values before you run ALTER TABLE • Be aware that the JSON type automatically uses the utf8mb4 character set • Morgan Tocker has a good blog post on this topic: http:// mysqlserverteam.com/upgrading-json-data-stored-in-text-columns/
How people build software! Reads versus writes Read/write balance considerations 56 !
How people build software! ! Read/write balance considerations 57 • New JSON type is read-optimized • Every update requires rewriting the entire object • Performance improvements are in progress for writes: • http://dev.mysql.com/worklog/task/?id=9141 • http://dev.mysql.com/worklog/task/?id=8985
How people build software! Disk storage implications 58 !
How people build software! ! Disk storage implications 59 • The JSON data type’s binary format uses about the same amount of space as text types. • Given the nature of JSON data (keys are repeated in every row), JSON data tends to compress well.
How people build software! Gotchas 60 !
How people build software! ! Gotchas 61 Namespace collisions between JSON UDFs and native functions: JSON_APPEND() JSON_DEPTH() JSON_EXTRACT() JSON_MERGE() JSON_REMOVE() JSON_REPLACE() JSON_SEARCH() JSON_SET() JSON_VALID()
How people build software! ! Gotchas 62 Stricter behavior of native functions: -- MySQL 5.6 UDF mysql> select json_extract('','foo'); +------------------------+ | json_extract('','foo') | +------------------------+ | NULL | +------------------------+ 1 row in set (0.00 sec) -- MySQL 5.7 native function mysql> select json_extract('','$.foo'); ERROR 3141 (22032): Invalid JSON text in argument 1 to function json_extract: "The document is empty." at position 0.
How people build software! ! Gotchas 63 Native functions output JSON, not text: -- MySQL 5.6 UDF mysql> select json_extract('{"foo":"bar"}','foo'); +-------------------------------------+ | json_extract('{"foo":"bar"}','foo') | +-------------------------------------+ | bar | +-------------------------------------+ 1 row in set (0.00 sec) -- MySQL 5.7 native function mysql> select json_extract('{"foo":"bar"}','$.foo'); +---------------------------------------+ | json_extract('{"foo":"bar"}','$.foo') | +---------------------------------------+ | "bar" | +---------------------------------------+ 1 row in set (0.12 sec)
How people build software! ! Gotchas 64 Path differences between JSON UDFs and native functions: -- MySQL 5.6 UDF mysql> select json_extract('{"parent":{"child":"hello"}}','parent','child'); +---------------------------------------------------------------+ | json_extract('{"parent":{"child":"hello"}}','parent','child') | +---------------------------------------------------------------+ | hello | +---------------------------------------------------------------+ 1 row in set (0.00 sec) -- MySQL 5.7 native function mysql> select json_extract('{"parent":{"child":"hello"}}','$.parent.child'); +---------------------------------------------------------------+ | json_extract('{"parent":{"child":"hello"}}','$.parent.child') | +---------------------------------------------------------------+ | "hello" | +---------------------------------------------------------------+ 1 row in set (0.00 sec)
How people build software! ! Gotchas 65 • Cannot index JSON columns directly: use generated columns for indexing
How people build software! ! Gotchas 66 Validating existing JSON values can be difficult if some rows are not valid JSON and other rows have a depth higher than 100: mysql> select json_text from old_table where json_valid(json_text) = 0; ERROR 3157 (22032): The JSON document exceeds the maximum depth. mysql> select id, json_depth(json_text) from old_table; ERROR 3141 (22032): Invalid JSON text in argument 1 to function json_depth: "Invalid value." at position 0.
How people build software! MySQL 8.0 67 !
How people build software! ! JSON changes in MySQL 8.0 68 • JSON aggregate functions: http://mysqlserverteam.com/mysql-8-0-labs-json- aggregation-functions/ • JSON_ARRAYAGG() • JSON_OBJECTAGG() • New function to prettify JSON values: https://dev.mysql.com/doc/refman/8.0/ en/json-utility-functions.html • JSON_PRETTY() • Storage used/free functions for inspecting native type • JSON_STORAGE_FREE() • JSON_STORAGE_SIZE() • “JSON in place” work underway • Optimizer piece complete • InnoDB BLOB refactoring complete
How people build software! ! Help shape the future!!! 69 • Proposal to change the behavior of JSON_MERGE: http:// mysqlserverteam.com/proposal-to-change-the-behavior-of-json_merge/ • How should JSON_MERGE() deal with overlapping values? • Current behavior = combine all values in an array • How should duplicate keys be handled? • Current behavior = first key wins • Common behavior = last key wins
How people build software! Thanks! Questions? Please rate my talk :) @iowalker My team is hiring! http://bit.ly/platform-data 70 !
How people build software ! "

More Related Content

What's hot (20)

PPTX
Six Degrees of Domain Admin - BloodHound at DEF CON 24
Andy Robbins
 
PPSX
Oracle Performance Tools of the Trade
Carlos Sierra
 
PPTX
High Performance, High Reliability Data Loading on ClickHouse
Altinity Ltd
 
PPTX
Metasploit
Lalith Sai
 
PDF
Nikto
Sorina Chirilă
 
PPTX
cluster computing
anjalibhandari11011995
 
PDF
Oracle 12c PDB insights
Kirill Loifman
 
PDF
Patroni - HA PostgreSQL made easy
Alexander Kukushkin
 
PDF
Access Control Models: Controlling Resource Authorization
Mark Niebergall
 
PDF
Rundeck Overview
Rundeck
 
PPTX
Cluster computing
pooja khatana
 
PDF
Connection Pooling in PostgreSQL using pgbouncer
Sameer Kumar
 
PDF
DBMS 10 | Database Transactions
Mohammad Imam Hossain
 
PDF
Linux File System
Anil Kumar Pugalia
 
PPT
Virtualization (Distributed computing)
Sri Prasanna
 
PDF
DBMS unit-1.pdf
Prof. Dr. K. Adisesha
 
PDF
New features in ProxySQL 2.0 (updated to 2.0.9) by Rene Cannao (ProxySQL)
Altinity Ltd
 
PDF
The Secret Of Hacking Trial Pages
leoimpact
 
PPTX
SQLite - Overview
Emanuele Bartolesi
 
PDF
Oracle RAC 19c: Best Practices and Secret Internals
Anil Nair
 
Six Degrees of Domain Admin - BloodHound at DEF CON 24
Andy Robbins
 
Oracle Performance Tools of the Trade
Carlos Sierra
 
High Performance, High Reliability Data Loading on ClickHouse
Altinity Ltd
 
Metasploit
Lalith Sai
 
Nikto
Sorina Chirilă
 
cluster computing
anjalibhandari11011995
 
Oracle 12c PDB insights
Kirill Loifman
 
Patroni - HA PostgreSQL made easy
Alexander Kukushkin
 
Access Control Models: Controlling Resource Authorization
Mark Niebergall
 
Rundeck Overview
Rundeck
 
Cluster computing
pooja khatana
 
Connection Pooling in PostgreSQL using pgbouncer
Sameer Kumar
 
DBMS 10 | Database Transactions
Mohammad Imam Hossain
 
Linux File System
Anil Kumar Pugalia
 
Virtualization (Distributed computing)
Sri Prasanna
 
DBMS unit-1.pdf
Prof. Dr. K. Adisesha
 
New features in ProxySQL 2.0 (updated to 2.0.9) by Rene Cannao (ProxySQL)
Altinity Ltd
 
The Secret Of Hacking Trial Pages
leoimpact
 
SQLite - Overview
Emanuele Bartolesi
 
Oracle RAC 19c: Best Practices and Secret Internals
Anil Nair
 

Similar to Practical JSON in MySQL 5.7 and Beyond (20)

PDF
Practical JSON in MySQL 5.7
Ike Walker
 
PPTX
The rise of json in rdbms land jab17
alikonweb
 
PDF
MySQL 5.7. Tutorial - Dutch PHP Conference 2015
Dave Stokes
 
PDF
MySQL 5.7 Tutorial Dutch PHP Conference 2015
Dave Stokes
 
PDF
Hybrid Databases - PHP UK Conference 22 February 2019
Dave Stokes
 
PDF
No sql way_in_pg
Vibhor Kumar
 
PDF
Webscale PostgreSQL - JSONB and Horizontal Scaling Strategies
Jonathan Katz
 
PPT
The NoSQL Way in Postgres
EDB
 
PPTX
From SQL to NoSQL: Structured Querying for JSON
Keshav Murthy
 
PPTX
Making MySQL Agile-ish
Dave Stokes
 
PDF
Using JSON with MariaDB and MySQL
Anders Karlsson
 
PDF
MySQL's JSON Data Type and Document Store
Dave Stokes
 
PPT
Do More with Postgres- NoSQL Applications for the Enterprise
EDB
 
PPSX
JSON in 18c and 19c
stewashton
 
PDF
JSON Processing in the Database using PostgreSQL 9.4 :: Data Wranglers DC :: ...
Ryan B Harvey, CSDP, CSM
 
PPSX
Json in 18c and 19c
stewashton
 
PPSX
JSON in Oracle 18c and 19c
stewashton
 
PPTX
Modern sql
Elizabeth Smith
 
PDF
Dealing with JSON in the relational world
Andres Almiray
 
PDF
Mathias test
Mathias Stjernström
 
Practical JSON in MySQL 5.7
Ike Walker
 
The rise of json in rdbms land jab17
alikonweb
 
MySQL 5.7. Tutorial - Dutch PHP Conference 2015
Dave Stokes
 
MySQL 5.7 Tutorial Dutch PHP Conference 2015
Dave Stokes
 
Hybrid Databases - PHP UK Conference 22 February 2019
Dave Stokes
 
No sql way_in_pg
Vibhor Kumar
 
Webscale PostgreSQL - JSONB and Horizontal Scaling Strategies
Jonathan Katz
 
The NoSQL Way in Postgres
EDB
 
From SQL to NoSQL: Structured Querying for JSON
Keshav Murthy
 
Making MySQL Agile-ish
Dave Stokes
 
Using JSON with MariaDB and MySQL
Anders Karlsson
 
MySQL's JSON Data Type and Document Store
Dave Stokes
 
Do More with Postgres- NoSQL Applications for the Enterprise
EDB
 
JSON in 18c and 19c
stewashton
 
JSON Processing in the Database using PostgreSQL 9.4 :: Data Wranglers DC :: ...
Ryan B Harvey, CSDP, CSM
 
Json in 18c and 19c
stewashton
 
JSON in Oracle 18c and 19c
stewashton
 
Modern sql
Elizabeth Smith
 
Dealing with JSON in the relational world
Andres Almiray
 
Mathias test
Mathias Stjernström
 
Ad

Recently uploaded (20)

PDF
Reverse Engineering of Security Products: Developing an Advanced Microsoft De...
nwbxhhcyjv
 
PDF
The Rise of AI and IoT in Mobile App Tech.pdf
IMG Global Infotech
 
PDF
Smart Trailers 2025 Update with History and Overview
Paul Menig
 
DOCX
Cryptography Quiz: test your knowledge of this important security concept.
Rajni Bhardwaj Grover
 
PPTX
From Sci-Fi to Reality: Exploring AI Evolution
Svetlana Meissner
 
PDF
What Makes Contify’s News API Stand Out: Key Features at a Glance
Contify
 
PPTX
AI Penetration Testing Essentials: A Cybersecurity Guide for 2025
defencerabbit Team
 
PDF
Jak MŚP w Europie Środkowo-Wschodniej odnajdują się w świecie AI
dominikamizerska1
 
PPTX
OpenID AuthZEN - Analyst Briefing July 2025
David Brossard
 
DOCX
Python coding for beginners !! Start now!#
Rajni Bhardwaj Grover
 
PDF
CIFDAQ Market Wrap for the week of 4th July 2025
CIFDAQ
 
PDF
Using FME to Develop Self-Service CAD Applications for a Major UK Police Force
Safe Software
 
PDF
Go Concurrency Real-World Patterns, Pitfalls, and Playground Battles.pdf
Emily Achieng
 
PDF
“NPU IP Hardware Shaped Through Software and Use-case Analysis,” a Presentati...
Edge AI and Vision Alliance
 
PDF
Newgen Beyond Frankenstein_Build vs Buy_Digital_version.pdf
darshakparmar
 
PPTX
The Project Compass - GDG on Campus MSIT
dscmsitkol
 
PDF
New from BookNet Canada for 2025: BNC BiblioShare - Tech Forum 2025
BookNet Canada
 
PDF
Newgen 2022-Forrester Newgen TEI_13 05 2022-The-Total-Economic-Impact-Newgen-...
darshakparmar
 
PDF
Transcript: New from BookNet Canada for 2025: BNC BiblioShare - Tech Forum 2025
BookNet Canada
 
PDF
"AI Transformation: Directions and Challenges", Pavlo Shaternik
Fwdays
 
Reverse Engineering of Security Products: Developing an Advanced Microsoft De...
nwbxhhcyjv
 
The Rise of AI and IoT in Mobile App Tech.pdf
IMG Global Infotech
 
Smart Trailers 2025 Update with History and Overview
Paul Menig
 
Cryptography Quiz: test your knowledge of this important security concept.
Rajni Bhardwaj Grover
 
From Sci-Fi to Reality: Exploring AI Evolution
Svetlana Meissner
 
What Makes Contify’s News API Stand Out: Key Features at a Glance
Contify
 
AI Penetration Testing Essentials: A Cybersecurity Guide for 2025
defencerabbit Team
 
Jak MŚP w Europie Środkowo-Wschodniej odnajdują się w świecie AI
dominikamizerska1
 
OpenID AuthZEN - Analyst Briefing July 2025
David Brossard
 
Python coding for beginners !! Start now!#
Rajni Bhardwaj Grover
 
CIFDAQ Market Wrap for the week of 4th July 2025
CIFDAQ
 
Using FME to Develop Self-Service CAD Applications for a Major UK Police Force
Safe Software
 
Go Concurrency Real-World Patterns, Pitfalls, and Playground Battles.pdf
Emily Achieng
 
“NPU IP Hardware Shaped Through Software and Use-case Analysis,” a Presentati...
Edge AI and Vision Alliance
 
Newgen Beyond Frankenstein_Build vs Buy_Digital_version.pdf
darshakparmar
 
The Project Compass - GDG on Campus MSIT
dscmsitkol
 
New from BookNet Canada for 2025: BNC BiblioShare - Tech Forum 2025
BookNet Canada
 
Newgen 2022-Forrester Newgen TEI_13 05 2022-The-Total-Economic-Impact-Newgen-...
darshakparmar
 
Transcript: New from BookNet Canada for 2025: BNC BiblioShare - Tech Forum 2025
BookNet Canada
 
"AI Transformation: Directions and Challenges", Pavlo Shaternik
Fwdays
 
Ad

Practical JSON in MySQL 5.7 and Beyond

  • 1. How people build software ! " Practical JSON in MySQL 5.7 and Beyond IkeWalker GitHub Percona Live April 27, 2017
  • 2. How people build software! What this talk is about 2 !
  • 3. How people build software! ! What this talk is about 3 • Using JSON in MySQL
  • 4. How people build software! What this talk is not about 4 !
  • 5. How people build software! ! What this talk is not about 5 • Whether you should use JSON in MySQL YES! NO!
  • 6. How people build software! A brief history of JSON in MySQL 6 !
  • 7. How people build software! ! Timeline: MySQL and JSON 7 1995: MYSQL 2002: JSON 2011: COMMON_SCHEMA 2013: MYSQL JSON UDFS 2015: NATIVE JSON
  • 8. How people build software! 2002 - 2011 “There’s an app for that” 8 !
  • 9. How people build software! ! 2002-2011: “There’s an app for that” 9 • Store JSON as text • Rewrite full string every time • Parsing happens exclusively in the application layer • Or write your own stored procedures/functions BUILD JSON WRITE TO DB STORE AS TEXT READ FROM DB PARSE JSON
  • 10. How people build software! 2011 - 2013 “Standard procedures” 10 !
  • 11. How people build software! ! 2011-2013: “Standard procedures” 11 • Store JSON as text • Rewrite full string every time • Some simple parsing can be done with common_schema: • get_option() • extract_json_value()
  • 12. How people build software! ! 2011-2013: “Standard procedures” 12 mysql> select common_schema.extract_json_value(f.event_data,'/age') as age, -> common_schema.extract_json_value(f.event_data,'/gender') as gender, -> sum(f.event_count) as event_count -> from json_event_fact f -> group by age, gender; +----------+---------+-------------+ | age | gender | event_count | +----------+---------+-------------+ | Over 30 | female | 3710983 | | Over 30 | male | 2869302 | | Under 30 | female | 5027591 | | Under 30 | male | 4918382 | | unknown | female | 42039 | | unknown | male | 50173 | | unknown | unknown | 8372 | +----------+---------+-------------+
  • 13. How people build software! 2013 - 2015 “Lab experiments” 13 !
  • 14. How people build software! ! 2013-2015: “Lab experiments” 14 • Store JSON as text • Some updates can be done with UDFs • Much faster parsing supported by UDFs
  • 15. How people build software! ! 2013-2015: “Lab experiments” 15 mysql> select json_extract(f.event_data,'state') as state, -> json_extract(f.event_data,'age') as age, -> json_extract(f.event_data,'gender') as gender, -> sum(f.event_count) as event_count -> from json_event_fact f -> group by state, age, gender; +-------+----------+--------+-------------+ | state | age | gender | event_count | +-------+----------+--------+-------------+ | CA | 18-24 | female | 5384 | | CA | 18-24 | male | 791 | | CA | 18-24 | null | 25 | | CA | 25-34 | female | 6731 | | CA | 25-34 | male | 1292 | | CA | 25-34 | null | 38 | | CA | 35-44 | female | 10638 | | CA | 35-44 | male | 1822 | | CA | 35-44 | null | 20 | +-------+----------+--------+-------------+
  • 16. How people build software! 2015 - present “Going native” 16 !
  • 17. How people build software! ! 2015-present: “Going native” 17 • Store JSON as text or JSON datatype • JSON datatype is binary, with keys sorted • Fast access to embedded data • Updates executed via native functions • Extensive parsing supported by native functions
  • 18. How people build software! JSON function examples 18 !
  • 19. How people build software! ! JSON_EXTRACT 19 mysql> select json_unquote(json_extract(event_data,'$.country’)) as country, -> sum(event_count) as events -> from json_event_fact -> where d = current_date() - interval 1 day -> and ad_id = 2 -> group by country; +---------+--------+ | country | events | +---------+--------+ | nl | 107954 | | us | 27373 | +---------+--------+ 2 rows in set (0.00 sec)
  • 20. How people build software! ! JSON_SEARCH 20 mysql> select json_search(event_data,'one','android') as json_path -> from json_event_fact -> having json_path is not null -> limit 1; +-----------+ | json_path | +-----------+ | "$.os" | +-----------+ 1 row in set (0.01 sec) mysql> select json_search('{"fa":"la","la":["la","la"]}','all','la')G *************************** 1. row *************************** json_search('{"fa":"la","la":["la","la"]}','all','la'): ["$.fa", "$.la[0]", "$.la[1]"] 1 row in set (0.00 sec)
  • 21. How people build software! ! JSON_REPLACE 21 mysql> set @json = cast('{"foo":"bar"}' as json); Query OK, 0 rows affected (0.00 sec) mysql> set @json = json_replace(@json, '$.foo', 'UPDATED'); Query OK, 0 rows affected (0.00 sec) mysql> select @json; +--------------------+ | @json | +--------------------+ | {"foo": "UPDATED"} | +--------------------+ 1 row in set (0.00 sec)
  • 22. How people build software! ! JSON_ARRAY 22 mysql> set @json = cast('{"foo":"bar"}' as json); Query OK, 0 rows affected (0.00 sec) mysql> set @json = json_replace(@json, '$.foo', json_array('bar', 'car', 'far')); Query OK, 0 rows affected (0.01 sec) mysql> select @json; +--------------------------------+ | @json | +--------------------------------+ | {"foo": ["bar", "car", "far"]} | +--------------------------------+ 1 row in set (0.00 sec)
  • 23. How people build software! ! JSON_OBJECT 23 mysql> set @json = json_object( -> 'id','007', -> 'name','James Bond', -> 'cars',json_array('Alfa Romeo','Aston Martin','BMW') -> ); Query OK, 0 rows affected (0.00 sec) mysql> select @jsonG *************************** 1. row *************************** @json: {"id": "007", "cars": ["Alfa Romeo", "Aston Martin", "BMW"], "name": "James Bond"} 1 row in set (0.00 sec)
  • 24. How people build software! ! JSON COMPARATOR 24 mysql> select cast('"hello"' as json) = 'hello'; 1 mysql> select cast(42 as json) = 42; 1 mysql> select cast(false as json) = false; 1 mysql> select cast('{"foo":"bar"}' as json) = cast('{"foo":"bar"}' as json); 1
  • 25. How people build software! ! But wait! There’s more! 25 https://dev.mysql.com/doc/refman/5.7/en/json-function-reference.html
  • 26. How people build software! Use case #1 Flexible rollups 26 !
  • 27. How people build software! ! Flexible rollups 27 • GOAL: Support different fields for multiple customers within a single dimension
  • 28. How people build software! ! Flexible rollups 28 For example, some ads track age and gender and others track country and os: {"age":"Over 30","gender":"female"} {"country":"us","os":"android"}
  • 29. How people build software! ! Flexible rollups 29 -- try to create rollup with JSON datatype mysql> create table json_event_fact ( -> d date not null, -> ad_id int not null, -> event_data json not null, -> event_count int not null, -> primary key (d,ad_id,event_data) -> ); ERROR 3152 (42000): JSON column 'event_data' cannot be used in key specification.
  • 30. How people build software! ! Flexible rollups 30 -- use text instead mysql> create table json_event_fact ( -> d date not null, -> ad_id int not null, -> event_data varchar(750) not null, -> event_count int not null, -> primary key (d,ad_id,event_data) -> ); Query OK, 0 rows affected (0.05 sec)
  • 31. How people build software! ! Flexible rollups 31 -- generated column hack mysql> create table json_event_fact ( -> d date not null, -> ad_id int not null, -> event_data json not null, -> event_data_text varchar(750) as (cast(event_data as char)) stored, -> event_count int not null, -> primary key (d,ad_id,event_data_text) -> ); Query OK, 0 rows affected (0.16 sec)
  • 32. How people build software! ! Flexible rollups 32 mysql> select event_data->'$.age' as age, -> sum(event_count) as events -> from json_event_fact -> where d = current_date() - interval 1 day -> and ad_id = 1 -> group by age; +------------+---------+ | age | events | +------------+---------+ | "Over 30" | 810424 | | "Under 30" | 1205544 | +------------+---------+ 2 rows in set (0.03 sec)
  • 33. How people build software! ! Flexible rollups 33 mysql> select json_unquote(event_data->'$.country') as country, -> sum(event_count) as events -> from json_event_fact -> where d = current_date() - interval 1 day -> and ad_id = 2 -> group by country; +---------+--------+ | country | events | +---------+--------+ | nl | 107954 | | us | 27373 | +---------+--------+ 2 rows in set (0.00 sec)
  • 34. How people build software! Use case #2 Configuration data 34 !
  • 35. How people build software! ! Configuration data 35 • GOAL: Store extensible configuration data of arbitrary depth for each ad
  • 36. How people build software! ! Configuration data 36 create table ad_config_data ( ad_config_data_id int not null primary key, ad_id int not null, name varchar(50) not null, config_data json not null );
  • 37. How people build software! ! Configuration data 37 { "a56a81eb": { "type": "AD", "uuid": "d9e9d8ae-8a33-11e6-97e0-22000b93579c", "subConfig": {}, "dataSupport": true }, "a6529578": { "type": "VIDEO", "uuid": "e09b40af-8a33-11e6-97e0-22000b93579c", "subConfig": { "video_url": "https://www.youtube.com/watch?v=dQw4w9WgXcQ", "hd": "true” } }, "a6caab6e": { "type": "AD", "uuid": "e89a3877-8a33-11e6-97e0-22000b93579c", "subConfig": {} } }
  • 38. How people build software! ! Configuration data 38 { "paths": { "path_3007ea93": ["action_312c40f4"], "path_30972a80": ["action_3158f2a7", "action_3185a6da", "action_31aedd9b"], … }, "actions": { "action_312c40f4": { … } } }
  • 39. How people build software! ! Configuration data 39 mysql> select json_extract(config_data,'$.paths.path_30c06190') as sub_paths from ad_config_data where ad_id = 1 and name = 'actions'G *************************** 1. row *************************** sub_paths: ["action_5b5343af", "action_5b8b6c39", "action_5bbb05d6"] 1 row in set (0.00 sec) mysql> update ad_config_data -> set config_data = json_replace(config_data,'$.paths.path_30c06190',json_array('action_5b5343af', 'action_5bbb05d6')) -> where ad_id = 1 and name = 'actions'; Query OK, 1 row affected (0.00 sec) Rows matched: 1 Changed: 1 Warnings: 0 mysql> select json_extract(config_data,'$.paths.path_30c06190') as sub_paths from ad_config_data where ad_id = 1 and name = 'actions'G *************************** 1. row *************************** sub_paths: ["action_5b5343af", "action_5bbb05d6"] 1 row in set (0.00 sec)
  • 40. How people build software! ! Configuration data 40 mysql> select json_extract(config_data,'$.*.type') -> from ad_config_data -> where ad_id = 1 -> and name = 'layers'; +--------------------------------------+ | json_extract(config_data,'$.*.type') | +--------------------------------------+ | ["AD", "VIDEO", "AD"] | +--------------------------------------+ 1 row in set (0.00 sec)
  • 41. How people build software! ! Configuration data 41 mysql> select json_search(config_data,'one','action_312c40f4') -> from ad_config_data -> where ad_id = 1 -> and name = 'actions'; +--------------------------------------------------+ | json_search(config_data,'one','action_312c40f4') | +--------------------------------------------------+ | "$.paths.path_3007ea93[0]" | +--------------------------------------------------+ 1 row in set (0.00 sec)
  • 42. How people build software! Use case #3 EAV antidote 42 !
  • 43. How people build software! ! EAV antidote 43 • GOAL: Store Entity-Attribute-Value style data while avoiding the EAV antipattern
  • 44. How people build software! ! EAV antidote 44 • The EAV antipattern is described well by Bill Karwin in his book: “SQL Antipatterns” • In Bill’s example the EAV anti-pattern is used to store two types of issues (bugs and features) in a single shared table.
  • 45. How people build software! ! EAV antidote 45 CREATE TABLE Issues ( issue_id SERIAL PRIMARY KEY ); CREATE TABLE IssueAttributes ( issue_id BIGINT UNSIGNED NOT NULL, attr_name VARCHAR(100) NOT NULL, attr_value VARCHAR(100), PRIMARY KEY (issue_id, attr_name), FOREIGN KEY (issue_id) REFERENCES Issues(issue_id) ); ATTRIBUTES STORED AS KEY-VALUE PAIRS
  • 46. How people build software! ! EAV antidote 46 CREATE TABLE Issues ( issue_id SERIAL PRIMARY KEY, reported_by BIGINT UNSIGNED NOT NULL, product_id BIGINT UNSIGNED, priority VARCHAR(20), version_resolved VARCHAR(20), status VARCHAR(20), issue_type VARCHAR(10), -- BUG or FEATURE attributes TEXT NOT NULL, -- all dynamic attributes for the row FOREIGN KEY (reported_by) REFERENCES Accounts(account_id), FOREIGN KEY (product_id) REFERENCES Products(product_id) );
  • 47. How people build software! ! EAV antidote 47 CREATE TABLE Issues ( issue_id SERIAL PRIMARY KEY, reported_by BIGINT UNSIGNED NOT NULL, product_id BIGINT UNSIGNED, priority VARCHAR(20), version_resolved VARCHAR(20), status VARCHAR(20), issue_type VARCHAR(10), -- BUG or FEATURE attributes JSON NOT NULL, -- all dynamic attributes for the row severity VARCHAR(20) AS (attributes->"$.severity"), -- only for bugs version_affected VARCHAR(20) AS (attributes->"$.version_affected"), -- only for bugs sponsor VARCHAR(50) AS (attributes->"$.sponsor"), -- only for feature requests FOREIGN KEY (reported_by) REFERENCES Accounts(account_id), FOREIGN KEY (product_id) REFERENCES Products(product_id) );
  • 48. How people build software! JSON + Generated Columns Two great techs that tech great together. 48 !
  • 49. How people build software! ! JSON + Generated Columns 49 • Allows you to expose one or more JSON fields as table columns • Supports indexes • Choose virtual (not stored) unless the index is Primary Key, FULLTEXT, or GIS
  • 50. How people build software! ! Example #1: Add age column to rollup table 50 mysql> alter table json_event_fact -> add column age varchar(20) as (event_data->'$.age'); Query OK, 0 rows affected (0.19 sec) Records: 0 Duplicates: 0 Warnings: 0 mysql> select age, -> sum(event_count) as events -> from json_event_fact -> where d = current_date() - interval 1 day -> and ad_id = 1 -> group by age; +------------+---------+ | age | events | +------------+---------+ | "Over 30" | 810424 | | "Under 30" | 1205544 | +------------+---------+ 2 rows in set (0.00 sec)
  • 51. How people build software! ! Example #2: Add gender column/index to rollup 51 mysql> alter table json_event_fact -> add column gender varchar(20) as (event_data->'$.gender'), -> add index (gender); Query OK, 0 rows affected (0.08 sec) Records: 0 Duplicates: 0 Warnings: 0 mysql> select sum(event_count) as events -> from json_event_fact -> where gender = '"female"'; +---------+ | events | +---------+ | 1081286 | +---------+ 1 row in set (0.00 sec)
  • 52. How people build software! JSON vs TEXT JSON as text versus JSON datatype 52 !
  • 53. How people build software! ! TEXT vs. JSON datatype 53 • Use the JSON datatype in general • There are a few exceptions. Use text types if you need to: • Include the column in a primary key • Store heterogeneous data (some rows are strings, some are JSON) • Store JSON with depth greater than 100
  • 54. How people build software! ! TEXT vs. JSON datatype 54 JSON data type sorts by keys: mysql> select cast('{"c":"3","b": {"2":"2","1":"1"},"a":"1"}' as json) as key_sort_test; +-------------------------------------------------+ | key_sort_test | +-------------------------------------------------+ | {"a": "1", "b": {"1": "1", "2": "2"}, "c": "3"} | +-------------------------------------------------+ 1 row in set (0.00 sec)
  • 55. How people build software! ! TEXT vs. JSON datatype 55 • Use the JSON_VALID() function to test validity of existing column values before you run ALTER TABLE • Be aware that the JSON type automatically uses the utf8mb4 character set • Morgan Tocker has a good blog post on this topic: http:// mysqlserverteam.com/upgrading-json-data-stored-in-text-columns/
  • 56. How people build software! Reads versus writes Read/write balance considerations 56 !
  • 57. How people build software! ! Read/write balance considerations 57 • New JSON type is read-optimized • Every update requires rewriting the entire object • Performance improvements are in progress for writes: • http://dev.mysql.com/worklog/task/?id=9141 • http://dev.mysql.com/worklog/task/?id=8985
  • 58. How people build software! Disk storage implications 58 !
  • 59. How people build software! ! Disk storage implications 59 • The JSON data type’s binary format uses about the same amount of space as text types. • Given the nature of JSON data (keys are repeated in every row), JSON data tends to compress well.
  • 60. How people build software! Gotchas 60 !
  • 61. How people build software! ! Gotchas 61 Namespace collisions between JSON UDFs and native functions: JSON_APPEND() JSON_DEPTH() JSON_EXTRACT() JSON_MERGE() JSON_REMOVE() JSON_REPLACE() JSON_SEARCH() JSON_SET() JSON_VALID()
  • 62. How people build software! ! Gotchas 62 Stricter behavior of native functions: -- MySQL 5.6 UDF mysql> select json_extract('','foo'); +------------------------+ | json_extract('','foo') | +------------------------+ | NULL | +------------------------+ 1 row in set (0.00 sec) -- MySQL 5.7 native function mysql> select json_extract('','$.foo'); ERROR 3141 (22032): Invalid JSON text in argument 1 to function json_extract: "The document is empty." at position 0.
  • 63. How people build software! ! Gotchas 63 Native functions output JSON, not text: -- MySQL 5.6 UDF mysql> select json_extract('{"foo":"bar"}','foo'); +-------------------------------------+ | json_extract('{"foo":"bar"}','foo') | +-------------------------------------+ | bar | +-------------------------------------+ 1 row in set (0.00 sec) -- MySQL 5.7 native function mysql> select json_extract('{"foo":"bar"}','$.foo'); +---------------------------------------+ | json_extract('{"foo":"bar"}','$.foo') | +---------------------------------------+ | "bar" | +---------------------------------------+ 1 row in set (0.12 sec)
  • 64. How people build software! ! Gotchas 64 Path differences between JSON UDFs and native functions: -- MySQL 5.6 UDF mysql> select json_extract('{"parent":{"child":"hello"}}','parent','child'); +---------------------------------------------------------------+ | json_extract('{"parent":{"child":"hello"}}','parent','child') | +---------------------------------------------------------------+ | hello | +---------------------------------------------------------------+ 1 row in set (0.00 sec) -- MySQL 5.7 native function mysql> select json_extract('{"parent":{"child":"hello"}}','$.parent.child'); +---------------------------------------------------------------+ | json_extract('{"parent":{"child":"hello"}}','$.parent.child') | +---------------------------------------------------------------+ | "hello" | +---------------------------------------------------------------+ 1 row in set (0.00 sec)
  • 65. How people build software! ! Gotchas 65 • Cannot index JSON columns directly: use generated columns for indexing
  • 66. How people build software! ! Gotchas 66 Validating existing JSON values can be difficult if some rows are not valid JSON and other rows have a depth higher than 100: mysql> select json_text from old_table where json_valid(json_text) = 0; ERROR 3157 (22032): The JSON document exceeds the maximum depth. mysql> select id, json_depth(json_text) from old_table; ERROR 3141 (22032): Invalid JSON text in argument 1 to function json_depth: "Invalid value." at position 0.
  • 67. How people build software! MySQL 8.0 67 !
  • 68. How people build software! ! JSON changes in MySQL 8.0 68 • JSON aggregate functions: http://mysqlserverteam.com/mysql-8-0-labs-json- aggregation-functions/ • JSON_ARRAYAGG() • JSON_OBJECTAGG() • New function to prettify JSON values: https://dev.mysql.com/doc/refman/8.0/ en/json-utility-functions.html • JSON_PRETTY() • Storage used/free functions for inspecting native type • JSON_STORAGE_FREE() • JSON_STORAGE_SIZE() • “JSON in place” work underway • Optimizer piece complete • InnoDB BLOB refactoring complete
  • 69. How people build software! ! Help shape the future!!! 69 • Proposal to change the behavior of JSON_MERGE: http:// mysqlserverteam.com/proposal-to-change-the-behavior-of-json_merge/ • How should JSON_MERGE() deal with overlapping values? • Current behavior = combine all values in an array • How should duplicate keys be handled? • Current behavior = first key wins • Common behavior = last key wins
  • 70. How people build software! Thanks! Questions? Please rate my talk :) @iowalker My team is hiring! http://bit.ly/platform-data 70 !
  • 71. How people build software ! "