Confoo 2021 - MySQL Indexes & Histograms

MySQL
Indexes,
Histograms,
And other ways
To Speed Up Your Queries
Dave Stokes @Stoker david.stokes @Oracle.com
Community Manager, North America
MySQL Community Team
Copyright © 2019 Oracle and/or its affiliates.
Confoo 2021

The following is intended to outline our general product direction. It is intended for information
purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any
material, code, or functionality, and should not be relied upon in making purchasing decisions. The
development, release, timing, and pricing of any features or functionality described for Oracle’s
products may change and remains at the sole discretion of Oracle Corporation.
Statements in this presentation relating to Oracle’s future plans, expectations, beliefs, intentions and
prospects are “forward-looking statements” and are subject to material risks and uncertainties. A
detailed discussion of these factors and other risks that affect our business is contained in Oracle’s
Securities and Exchange Commission (SEC) filings, including our most recent reports on Form 10-K
and Form 10-Q under the heading “Risk Factors.” These filings are available on the SEC’s website
or on Oracle’s website at http://www.oracle.com/investor. All information in this presentation is
current as of September 2019 and Oracle undertakes no duty to update any statement in light of
new information or future events.
Safe Harbor
Copyright © 2019 Oracle and/or its affiliates.

Warning!
MySQL 5.6 End of Life is
February of 2021!! 3
WAS

Test Drive MySQL
Database Service For
Free Today
Get $300 in credits
and try MySQL Database Service
free for 30 days.
https://www.oracle.com/cloud/free/
Copyright © 2020, Oracle and/or its affiliates | Confidential: Internal/Restricted/Highly Restricted
4

What Is This Session About?
Nobody ever complains that the database is too fast!
Speeding up queries is not a ‘dark art’
But understanding how to speed up queries is often
treated as magic
So we will be looking at the proper use of indexes,
histograms, locking options, and some other ways to speed
queries up.
5

Yes, this is a very dry subject!
● How dry?
○ Very dry
● Lots of text on screen
○ Download slides and use as a reference later
■ slideshare.net/davidmstokes
○ Do not try to absorb all at once
■ Work on most frequent query first, then second most, …
■ And optimizations may need to change over time
● No Coverage today of
○ System configuration
■ OS
■ MySQL
○ Hardware
○ Networking Slides are posted to https://slideshare.net/davestokes 6

Normalize Your Data (also not covered today)
● Can not build a skyscraper on a foundation of sand
● Third normal form or better
○ Use JSON for ‘stub’ table data, avoiding
repeated unneeded index/table accesses
● Think about how you will use your data
○ Do not use a fork to eat soup; How do you consume your data
Badly normalized data will hurt the performance of your queries. No matter
how much training you give it, a dachshund will not be faster than a thoroughbred
horse!
7

The Optimizer
● Consider the optimizer the brain and
nervous system of the system.
● Query optimization is a feature of
many relational database
management systems.
● The query optimizer attempts to
determine the most efficient way to
execute a given query by considering
the possible query plans. Wikipedia
8
One of the hardest problems in query optimization is to
accurately estimate the costs of alternative query plans.
Optimizers cost query plans using a mathematical model
of query execution costs that relies heavily on estimates
of the cardinality, or number of tuples, flowing through
each edge in a query plan.
Cardinality estimation in turn depends on estimates of
the selection factor of predicates in the query.
Traditionally, database systems estimate selectivities
through fairly detailed statistics on the distribution of
values in each column, such as histograms.

The query optimizer evaluates the options
● The optimizer wants to get your data the
cheapest way (least amount of very
expensive disk reads) possible.
● Like a GPS, the cost is built on historical
statistics. And these statistics can
change while the optimizer is working. So
like a traffic jam, washed out road, or other
traffic problem, the optimizer may be
making poor decisions for the present
situation.
● The final determination from the optimizer
is call the query plan.
● MySQL wants to optimize each query
every time it sees it – there is no locking
down the query plan like Oracle.
{watch for optimizer hints later in this
presentation}
You will see how to obtain a query plan
later in this presentation.
9

120!
If your query has five joins then the optimizer
may have to evaluate 120 different options
5!
(5 * 4 * 3 * 2 * 1)
10

EXPLAIN
Explaining EXPLAIN requires a great deal of explanation
11

EXPLAIN Syntax
12
Query optimization is covered in
Chapter 8 of the MySQL Manual
EXPLAIN reports how the
server would process the
statement, including information
about how tables are joined and
in which order.

But now for something completely different
The tools for looking at queries
● EXPLAIN
○ EXPLAIN FORMAT=
■ JSON
■ TREE
○ ANALYZE
● VISUAL EXPLAIN 13

EXPLAIN Example
14
EXPLAIN is used to obtain a query execution plan (an explanation of how MySQL would
execute a query) and should be considered an ESTIMATE as it does not run the query.
QUERY
QUERY PLAN
DETAILS

VISUAL EXPLAIN from MySQL Workbench
15

EXPLAIN FORMAT=TREE
EXPLAIN FORMAT=TREE SELECT * FROM City
JOIN Country ON (City.Population = Country.Population);
-> Inner hash join (city.Population = country.Population) (cost=100154.05 rows=100093)
-> Table scan on City (cost=0.30 rows=4188)
-> Hash
-> Table scan on Country (cost=29.90 rows=239)
16

EXPLAIN FORMAT=JSON
EXPLAIN FORMAT=JSON SELECT * FROM City JOIN Country ON
(City.Population = Country.Population)G
*************************** 1. row
***************************
EXPLAIN: {
"query_block": {
"select_id": 1,
"cost_info": {
"query_cost": "100154.05"
},
"nested_loop": [
{
"table": {
"table_name": "Country",
"access_type": "ALL",
"rows_examined_per_scan": 239,
"rows_produced_per_join": 239,
"filtered": "100.00",
"cost_info": {
"read_cost": "6.00",
"eval_cost": "23.90",
"prefix_cost": "29.90",
"data_read_per_join": "61K"
},
17
"used_columns": [
"Code",
"Name",
"Continent",
"Region",
"SurfaceArea",
"IndepYear",
"Population",
"LifeExpectancy",
"GNP",
"GNPOld",
"LocalName",
"GovernmentForm",
"HeadOfState",
"Capital",
"Code2"
]
}
},
{
"table": {
"table_name": "City",
"access_type": "ALL",
"rows_examined_per_scan": 4188,
"rows_produced_per_join": 100093,
"filtered": "10.00",
"using_join_buffer": "hash join",
"cost_info": {
"read_cost": "30.95",
"eval_cost": "10009.32",
"prefix_cost": "100154.05",
"data_read_per_join": "6M"
},
"used_columns": [
"ID",
"Name",
"CountryCode",
"District",
"Population"
],
"attached_condition": "(`world`.`city`.`Population` =
`world`.`country`.`Population`)"
}
}
]
}
}

EXPLAIN ANALYZE – MySQL 8.0.18
18
EXPLAIN ANALYZE SELECT * FROM City WHERE CountryCode = 'GBR'G
*************************** 1. row ***************************
EXPLAIN: -> Index lookup on City using CountryCode (CountryCode='GBR')
(cost=80.76 rows=81) (actual time=0.153..0.178 rows=81 loops=1)
1 row in set (0.0008 sec)
MySQL 8.0.18 introduced EXPLAIN ANALYZE, which runs the query and produces
EXPLAIN output along with timing and additional, iterator-based, information about how the
optimizer's expectations matched the actual execution. Real numbers not an estimate!

More on using
EXPLAIN …
later 19

Indexes
A database index is a data structure that improves the speed of data retrieval
operations on a database table at the cost of additional writes and storage space
to maintain the index data structure.
Indexes are used to quickly locate data without having to search every row in a
database table every time a database table is accessed.
Indexes can be created using one or more columns of a database table, providing
the basis for both rapid random lookups and efficient access of ordered records. -
- https://en.wikipedia.org/wiki/Database_index
21

Think of an Index as a Table
With Shortcuts to another
table!
Or a model of some of your data in it’s own table!
And the more tables you have the read to get to
the data the slower things run!
(And the more memory is taken up) 22

25
clustered index
The InnoDB term for a primary key index. InnoDB table storage is organized based on the values of the primary key columns,
to speed up queries and sorts involving the primary key columns.
For best performance, choose the primary key columns carefully based on the most performance-critical queries.
Because modifying the columns of the clustered index is an expensive operation, choose primary columns that are rarely or
never updated.
In the Oracle Database product, this type of table is known as an index-organized table

Think of an index as a small, fast table pointing to a bigger table
26
Index
3
6
12
27
5001
Indexed Column column 1 column x column n
5001
3
6
27
12

How Secondary Indexes Relate to the Clustered Index
All indexes other than the clustered index are known as secondary indexes.
In InnoDB, each record in a secondary index contains the primary key columns for the row, as well as
the columns specified for the secondary index. InnoDB uses this primary key value to search for the row
in the clustered index.
If the primary key is long, the secondary indexes use more space, so it is advantageous to have a short
primary key.
27

28
Creating a table with a PRIMARY KEY (index)
CREATE TABLE t1 (
c1 INT NOT NULL AUTO_INCREMENT PRIMARY KEY,
c2 VARCHAR(100),
c3 VARCHAR(100) );
An index is a list of KEYs.
And you will hear ‘key’ and ‘index’ used interchangeably

PRIMARY KEY
This is an key for the index that uniquely defined for a row, should be immutable.
InnoDB needs a PRIMARY KEY (and will make one up if you do not specify)
No Null Values
Monotonically increasing
- use UUID_To_BIN() if you must use UUIDS, otherwise avoid them
29

Indexing on a prefix of a column
CREATE INDEX part_of_name ON customer (name(10));
Only the first 10 characters are indexed in this examples and
this can save space/speed
30

Multi-column index - last_name, first_name
CREATE TABLE test (
id INT NOT NULL,
last_name CHAR(30) NOT NULL,
first_name CHAR(30) NOT NULL,
PRIMARY KEY (id),
INDEX name (last_name,first_name)
);
31
This index will work on (last_name,first_name) and (last_name) but not (first_name)
{Use: left to right}
Put highest cardinality field first

Hashing values - an alternative
SELECT * FROM tbl_name
WHERE hash_col=MD5(CONCAT(val1,val2))
AND col1=val1 AND col2=val2;
As an alternative to a composite index, you can introduce a
column that is “hashed” based on information from other columns.
If this column is short, reasonably unique, and indexed, it
might be faster than a “wide” index on many columns.
32

Some other types of indexes
Unique Indexes - only one row per value
Full-Text Indexes - search string data
Covering index - includes all columns needed for a query
Secondary index - another column in the table is indexes
Spatial Indexes - geographical information
33

Functional Indexes are defined on the result of a function
applied to one or more columns of a single table
CREATE TABLE t1 (col1 INT, col2 INT, INDEX func_index ((ABS(col1))));
CREATE INDEX idx1 ON t1 ((col1 + col2));
CREATE INDEX idx2 ON t1 ((col1 + col2), (col1 - col2), col1);
ALTER TABLE t1 ADD INDEX ((col1 * 40) DESC);
34

Multi-value indexes
You can now have more index pointers than index keys!
○ Very useful for JSON arrays
mysql> SELECT 3 MEMBER OF('[1, 3, 5, 7, "Moe"]');
+--------------------------------------+
| 3 MEMBER OF('[1, 3, 5, 7, "Moe"]') |
+--------------------------------------+
| 1 |
+--------------------------------------+
35

MySQL has Two Main Types of Index Structures
B-Tree is a self-balancing tree data structure
that maintains sorted data and allows searches,
sequential access, insertions, and deletions.
36
Hash are more efficient than nested loops joins,
except when the probe side of the join is very
small but joins can only be used to compute
equijoins.

Please Keep in mind ...
If there is a choice between multiple indexes, MySQL normally uses the index that
finds the smallest number of rows (the most selective index).
MySQL can use indexes on columns more efficiently if they are declared as the
same type and size.
● In this context, VARCHAR and CHAR are considered the same if they are
declared as the same size. For example, VARCHAR(10) and CHAR(10) are
the same size, but VARCHAR(10) and CHAR(15) are not.
For comparisons between nonbinary string columns, both columns should use the
same character set. For example, comparing a utf8 column with a latin1 column
precludes use of an index. 37

NULL
A few words on something that may not be there!
38

NULL
● NULL is used to designate a
LACK of data.
False 0
True 1
Don’t know NULL
39

Indexing NULL values
really drives down the
performance of
INDEXes -
Cardinal Values and then a ‘junk drawer’ of nulls -> hard to search
40

Before Invisible Indexes
1. Doubt usefulness of index
2. Check using EXPLAIN
3. Remove that Index
4. Rerun EXPLAIN
5. Get phone/text/screams from power user about slow query
6. Suddenly realize that the index in question may have had no use for you but
the rest of the planet seems to need that dang index!
7. Take seconds/minutes/hours/days/weeks rebuilding that index
42

After Invisible Indexes
1. Doubt usefulness of index
2. Check using EXPLAIN
3. Make index invisible – optimizer can not see that index!
4. Rerun EXPLAIN
5. Get phone/text/screams from power user about slow query
6. Make index visible
7. Blame problem on { network | JavaScript | GDPR | Slack | Cloud }
Sys schema will show you which indexes have not been used that may be
candidates for removal -- but be cautious!
43

How to use INVISIBLE INDEX
ALTER TABLE t1 ALTER INDEX i_idx INVISIBLE;
ALTER TABLE t1 ALTER INDEX i_idx VISIBLE;
44

Histograms
What is a histogram?
It is not a gluten free, keto friendly biscuit.
45

Histograms
Wikipedia declares a histogram is an accurate representation of the distribution of
numerical data. For RDBMS, a histogram is an approximation of the data
distribution within a specific column.
So in MySQL, histograms help the optimizer to find the most efficient Query Plan
to fetch that data.
47

Histograms
A histogram is a distribution of data into logical buckets.
There are two types:
● Singleton
● Equi-Height
Maximum number of buckets is 1024
48

Histogram
Histogram statistics are useful primarily for non-indexed columns.
A histogram is created or updated only on demand, so it adds no
overhead when table data is modified. On the other hand, the statistics become
progressively more out of date when table modifications occur, until the next time
they are updated.
49

The two reasons for why you might consider a histogram instead of an index:
Maintaining an indexes have a cost. If you have an index, every
● Every INSERT/UPDATE/DELETE causes the index to be updated. This will
have an impact on your performance. A histogram on the other hand is
created once and never updated unless you explicitly ask for it. It will thus not
hurt your INSERT/UPDATE/DELETE-performance.
● The optimizer will make “index dives” to estimate the number of records in a
given range. This might become too costly if you have for instance very long
IN-lists in your query. Histogram statistics are much cheaper in this case, and
might thus be more suitable.
50

The Optimizer
Occasionally the query optimizer fails to find the most efficient plan and ends up
spending a lot more time executing the query than necessary.
The optimizer assumes that the data is evenly distributed in the column. This can
be the old ‘assume’ makes an ‘ass out of you and me’ joke brought to life.
The main reason for this is often that the optimizer doesn’t have enough
knowledge about the data it is about to query:
● How many rows are there in each table?
● How many distinct values are there in each column?
● How is the data distributed in each column?
51

Two-types of histograms
Equi-height: One bucket represents a range of values. This type of histogram will
be created when distinct values in the column are greater than the number of
buckets specified in the analyze table syntax. Think A-G H-L M-T U-Z
Singleton: One bucket represents one single value in the column and is the most
accurate and will be created when the number of distinct values in the column is
less than or equal to the number of buckets specified in the analyze table syntax.
52

Frequency
Histogram
Each distinct value
has its own bucket.
53
101
102 104
insert into freq_histogram (x)
values
(101),(101),(102),(102),(102),(104);

Frequency Histogram - an Example
select id,x from freq_histogram ;
+----+-----+
| id | x |
+----+-----+
| 1 | 101 |
| 2 | 101 |
| 3 | 102 |
| 4 | 102 |
| 5 | 102 |
| 6 | 104 |
+----+-----+
analyze table freq_histogram
update histogram on x with 3 buckets;
54
select JSON_PRETTY(HISTOGRAM->>"$") from
information_schema.column_statistics where
table_name='freq_histogram' and column_name='x'G
*************************** 1. row
***************************
JSON_PRETTY(HISTOGRAM->>"$"): {
"buckets": [
[
101,
0.3333333333333333
],
[
102,
0.8333333333333333
],
[
104,
1.0
]
],
"data-type": "int",
"null-values": 0.0,
"collation-id": 8,
"last-updated": "2020-06-16 19:54:21.033017",
"sampling-rate": 1.0,
"histogram-type": "singleton",
"number-of-buckets-specified": 3
}

The statistics
SELECT (SUBSTRING_INDEX(v, ':', -1)) value,
concat(round(c*100,1),'%') cumulfreq,
CONCAT(round((c - LAG(c, 1, 0) over()) * 100,1), '%') freq
FROM information_schema.column_statistics,
JSON_TABLE(histogram->'$.buckets','$[*]'
COLUMNS(v VARCHAR(60) PATH '$[0]',
c double PATH '$[1]')) hist
WHERE schema_name = 'demo'
and table_name = 'freq_histogram'
and column_name = 'x';
+-------+-----------+-------+
| value | cumulfreq | freq |
+-------+-----------+-------+
| 101 | 33.3% | 33.3% |
| 102 | 83.3% | 50.0% |
| 104 | 100.0% | 16.7% |
+-------+-----------+-------+
55

Creating and removing Histograms
ANALYZE TABLE t UPDATE HISTOGRAM ON c1, c2, c3 WITH 10 BUCKETS;
ANALYZE TABLE t UPDATE HISTOGRAM ON c1, c3 WITH 10 BUCKETS;
ANALYZE TABLE t DROP HISTOGRAM ON c2;
Note the first statement creates three different histograms on c1, c2, and c3.
56

Where Histograms Shine
create table h1 (id int unsigned auto_increment,
x int unsigned, primary key(id));
insert into h1 (x) values (1),(1),(2),(2),(2),(3),(3),(3),(3);
select x, count(x) from h1 group by x;
+---+----------+
| x | count(x) |
+---+----------+
| 1 | 2 |
| 2 | 3 |
| 3 | 4 |
+---+----------+
3 rows in set (0.0011 sec)
58

Without Histogram
EXPLAIN SELECT * FROM h1 WHERE x > 0G
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: h1
partitions: NULL
type: ALL
possible_keys: NULL
key: NULL
key_len: NULL
ref: NULL
rows: 9
filtered: 33.32999801635742 – The optimizer estimates about 1/3 of the data
Extra: Using where will go match the ‘X > 0’ condition
1 row in set, 1 warning (0.0007 sec) 59
The filtered column indicates an estimated percentage of table rows that will
be filtered by the table condition. The maximum value is 100, which means no
filtering of rows occurred.
Values decreasing from 100 indicate increasing amounts of filtering. rows shows
the estimated number of rows examined and rows × filtered shows the
number of rows that will be joined with the following table.
For example, if rows is 1000 and filtered is 50.00 (50%), the number of rows
to be joined with the following table is 1000 × 50% = 500.

ANALYZE TABLE h1 UPDATE HISTOGRAM ON x WITH 3 BUCKETSG
*************************** 1. row ***************************
Table: demox.h1
Op: histogram
Msg_type: status
Msg_text: Histogram statistics created for column 'x'.
60

EXPLAIN SELECT * FROM h1 WHERE x > 0G
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: h1
partitions: NULL
type: ALL
possible_keys: NULL
key: NULL
key_len: NULL
ref: NULL
rows: 9
filtered: 100 – all rows!!!
Extra: Using where
1 row in set, 1 warning (0.0007 sec) 61

With EXPLAIN ANALYSE
EXPLAIN analyze SELECT * FROM h1 WHERE x > 0G
*************************** 1. row ***************************
EXPLAIN: -> Filter: (h1.x > 0) (cost=1.15 rows=9) (actual time=0.027..0.034 rows=9 loops=1)
-> Table scan on h1 (cost=1.15 rows=9) (actual time=0.025..0.030 rows=9 loops=1)
62

Performance is not just Indexes and Histograms
● There are many other ‘tweaks’ that can be made to speed things up
● Use explain to see what your query is doing?
○ File sorts, full table scans, using temporary tables, etc.
○ Does the join order look right
○ Buffers and caches big enough
○ Do you have enough memory
○ Disk and I/O speeds sufficient
63

Locking Options
MySQL added two locking options to MySQL 8.0:
● NOWAIT
○ A locking read that uses NOWAIT never waits to acquire a row lock. The query executes
immediately, failing with an error if a requested row is locked.
● SKIP LOCKED
○ A locking read that uses SKIP LOCKED never waits to acquire a row lock. The query executes
immediately, removing locked rows from the result set.
64

Locking Examples – Buying concert tickets
START TRANSACTION;
SELECT seat_no, row_no, cost
FROM seats s
JOIN seat_rows sr USING ( row_no )
WHERE seat_no IN ( 3,4 ) AND sr.row_no IN ( 5,6
)
AND booked = 'NO'
FOR UPDATE OF s SKIP LOCKED;
Let’s shop for tickets in rows 5 or 6 and seats 3 & 4 but we skip any locked
records!
65

LOCK NOWAIT
START TRANSACTION;
SELECT seat_no
FROM seats JOIN seat_rows USING ( row_no )
WHERE seat_no IN (3,4) AND seat_rows.row_no IN (12)
AND booked = 'NO'
FOR UPDATE OF seats SKIP LOCKED
FOR SHARE OF seat_rows NOWAIT;
Without NOWAIT, this query would have waited for innodb_lock_wait_timeout (default: 50)
seconds while attempting to acquire the shared lock on seat_rows. With NOWAIT an error is
immediately returned ERROR 3572 (HY000): Do not wait for lock. 66

Other Fast Ways
Resource Groups
Optimizer hints
Partitioning
Multi Value Indexes
67

Resource groups – setting and using
CREATE RESOURCE GROUP Batch
TYPE = USER VCPU = 2-3 -- assumes a system with at least 4 CPUs
THREAD_PRIORITY = 10;
SET RESOURCE GROUP Batch;
or
INSERT /*+ RESOURCE_GROUP(Batch) */ INTO t2 VALUES(2);
68

Optimizer Hints
SELECT /*+ JOIN_ORDER(t1, t2) */ ... FROM t1, t2;
Optimizer hints can be specified within individual statements. Because the
optimizer hints apply on a per-statement basis, they provide finer control over
statement execution plans than can be achieved using optimizer_switch.
For example, you can enable an optimization for one table in a statement and
disable the optimization for a different table.
69

Partitioning
● In MySQL 8.0, partitioning support is provided by the InnoDB and NDB
storage engines.
● Partitioning enables you to distribute portions of individual tables across a file
system according to rules which you can set largely as needed. In effect,
different portions of a table are stored as separate tables in different
locations.
70

The Query
EXPLAIN SELECT City.name,
Country.Name
FROM City
JOIN Country ON
(City.CountryCode = Country.Code)
WHERE Country.Code = 'GBR'G
71

The Query – What we want
Country.Name
FROM City
JOIN Country ON (City.CountryCode =
Country.Code)
72

The Query – Where we want it from
Country.Name
FROM City
JOIN Country ON
73

The Query – How the two tables relate
Country.Name
FROM City
JOIN Country ON
74

The Query – And any filters
Country.Name
FROM City
JOIN Country ON
75

76
EXPLAIN SELECT City.name, Country.Name FROM City
JOIN Country ON (City.CountryCode = Country.Code) WHERE Country.Code = 'GBR'G
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: Country
partitions: NULL
type: const
possible_keys: PRIMARY
key: PRIMARY
key_len: 3
ref: const
rows: 1
filtered: 100
Extra: NULL
*************************** 2. row ***************************
id: 1
select_type: SIMPLE
table: City
partitions: NULL
type: ref
possible_keys: CountryCode
key: CountryCode
key_len: 3
ref: const
rows: 81
filtered: 100
Extra: NULL
2 rows in set, 1 warning (0.0013 sec)

The Actual Query Plan
select
`world`.`city`.`Name` AS `name`,
'United Kingdom' AS `Name`
from `world`.`city`
join `world`.`country` where
(`world`.`city`.`CountryCode` = 'GBR')
77

Optimizer substituted Country.name
select
`world`.`city`.`Name` AS `name`,
'United Kingdom' AS `Name`
from `world`.`city`
join `world`.`country` where
(`world`.`city`.`CountryCode` = 'GBR')
78

explain format=tree SELECT City.name, Country.Name FROM City
JOIN Country ON (City.CountryCode = Country.Code) WHERE
Country.Code = 'GBR'G
*************************** 1. row ***************************
EXPLAIN: -> Index lookup on City using CountryCode
(CountryCode='GBR') (cost=26.85 rows=81)
So the optimizer ‘knows’ just to grab the ‘GBR’ records from the City table and does
not need to read the Country table at all.
79

Explain ANALYZE SELECT City.name, Country.Name FROM City JOIN
Country ON (City.CountryCode = Country.Code) WHERE Country.Code
= 'GBR'G
*************************** 1. row ***************************
EXPLAIN: -> Index lookup on City using CountryCode
(CountryCode='GBR') (cost=26.85 rows=81)
(actual time=0.123..0.138 rows=81 loops=1)
Explain ANALYZE runs the query and the estimate is pretty good!
80

81
EXPLAIN SELECT City.name, Country.Name
FROM City
JOIN Country ON (City.CountryCode = Country.Code) WHERE Country.Code = 'GBR'G
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: Country
partitions: NULL
type: const
possible_keys: PRIMARY
key: PRIMARY
key_len: 3
ref: const This is a CONSTANT
rows: 1
filtered: 100
Extra: NULL
*************************** 2. row ***************************
id: 1
select_type: SIMPLE
table: City
partitions: NULL
type: ref
possible_keys: CountryCode
key: CountryCode
key_len: 3
ref: const
rows: 81 There are 81 records that match in the index
filtered: 100
Extra: NULL
2 rows in set, 1 warning (0.0013 sec)

Some General Rules
● Look at indexing/histogramming columns on right side of WHERE clause
● Maybe index SORT BY columns (test)
● JOIN on like type and size columns
○ i.e. No VARCHAR(32) to DECIMAL matches
● Can the data be found in an index?
83

Slightly More Complex
SELECT CONCAT(customer.last_name,', ', customer.first_name) AS customer,
address.phone,
film.title
FROM rental
INNER JOIN customer ON rental.customer_id = customer.customer_id
INNER JOIN address ON customer.address_id = address.address_id
INNER JOIN inventory ON rental.inventory_id = inventory.inventory_id
INNER JOIN film ON inventory.film_id = film.film_id
WHERE rental.return_date IS NULL
AND rental_date + INTERVAL film.rental_duration DAY < CURRENT_DATE()
ORDER BY title
LIMIT 5;
84

Tree format is often little better
Limit: 5 row(s)
-> Sort: <temporary>.title, limit input to 5 row(s) per chunk
-> Stream results
-> Nested loop inner join (cost=3866.17 rows=1601)
-> Filter: (rental.return_date is null) (cost=1625.05 rows=1601)
-> Table scan on rental (cost=1625.05 rows=16008)
-> Single-row index lookup on customer using PRIMARY (customer_id=rental.customer_id) (cost=0.25 rows=1)
-> Single-row index lookup on address using PRIMARY (address_id=customer.address_id) (cost=0.25 rows=1)
-> Single-row index lookup on inventory using PRIMARY (inventory_id=rental.inventory_id) (cost=0.25 rows=1)
-> Filter: ((rental.rental_date + interval film.rental_duration day) < <cache>(curdate())) (cost=0.25 rows=1)
-> Single-row index lookup on film using PRIMARY (film_id=inventory.film_id) (cost=0.25 rows=1)
87

Slightly More Complex
address.phone,
film.title
FROM rental
ORDER BY title
LIMIT 5;
91

Quiz: Why so slow if it only has to return FIVE RECORDS?!?!?!?!
address.phone,
film.title
FROM rental
ORDER BY title
LIMIT 5;
92

Second glance – what EXTRAS are being used?
|
+----+-------------+-----------+------------+--------+----------------------------------------+---------+---------+----------------------------
+-------+----------+----------------------------------------------+
|
|
|
| film | eq_ref | PRIMARY 100 | NULL |
| film | eq_ref | PRIMARY
93
The where; Using temporary; Using filesort informs that the output
has to be sorted (part of the sort clause) and that a temporary
table needed to be used.

MySQL 8.0 Temporary Table much faster
● Previously temporary tables were size limited and when they
reached that limit:
○ The processing was halted
○ The data was copied to InnoDB
○ The processing continued with the InnoDB copy of the data
○ And the above was slow
94

Is there something unindexed that need to be indexed?
● Or something we can use to build a histogram?
● Is there a better way to make the key?
95

SHOW INDEX FROM rental
● PRIMARY
● Rental_date – rental_date, inventory_id, customer_id
● Idx_fk_inventory_id – inventory_id, cutomer_id
● Idx_fk_staff_id
96

SHOW INDEX FROM film
● PRIMARY
● Idx_title
● Idx_fk_language_id
● Idx_fk_original_language_id
97

A Functional Index? Or Generated Column?
AND rental_date + INTERVAL film.rental_duration DAY < CURRENT_DATE() ORDER BY title LIMIT 5;
Could this part in red be reduced to a functional index/generated column?
AND return_date < CURRENT_DATE()
film.rental_duration + rental_date
Would want to store this in the record with the rental data?
98

Yes, let us add that new column
● ALTER TABLE operations can be expensive
● How do we seed data
● Do we use a GENERATED COLUMN? Can we?
● Do we add a stub table?
○ Extra index/table dives
○ How much code do we have to change?
○ Other considerations
99

Where Else To Look for Information
● MySQL Manual
● Forums.MySQL.com
● MySQLcommunity.slack.com
100

New Book
● YOU NEED THESE BOOKS!!
● Author’s others work are also
outstanding and well written
● Amazing detail!!
101

Great Book
● Getting very dated
● Make sure you get the 3rd edition
● Can be found in used book stores
102

My Book
● A guide to using JSON and MySQL
○ Programming Examples
○ Best practices
○ Twice the length of the original
○ NoSQL Document Store how-to
103

“We have saved around 40% of our
costs and are able to reinvest that
back into the business. And we are
scaling across EMEA, and that’s
basically all because of Oracle.”
—Asser Smidt
CEO and Cofounder, BotSupply
Startups get cloud credits and a 70%
discount for 2 years, global exposure via
marketing, events, digital promotion, and
media, plus access to mentorship, capital
and Oracle’s 430,000+ customers
Customers meet vetted startups in
transformative spaces that help them stay
ahead of their competition
Oracle stays at the competitive edge
of innovation with solutions that complement
its technology stack
Oracle for Startups - enroll at oracle.com/startup
A Virtuous Cycle of Innovation, Everybody Wins.

Thank you
● David.Stokes @Oracle.com
● @Stoker
● PHP Related blog https://elephantdolphin.blogspot.com/
● MySQL Latest Blogs https://planet.mysql.com/
● Where to ask questions https://forums.mysql.com/
● Slack https://mysqlcommunity.slack.com
slides -> slideshare.net/davestokes
105

Confoo 2021 - MySQL Indexes & Histograms

Recommended

More Related Content

What's hot (20)

Similar to Confoo 2021 - MySQL Indexes & Histograms (20)

More from Dave Stokes (14)

Recently uploaded (20)

Confoo 2021 - MySQL Indexes & Histograms