01 upgrade to my sql8

Ted Wennmark
MySQL Solution Architect EMEA
Blog : http://mysql-nordic.blogspot.com/
Git projects : https://github.com/wwwted
LinkedIn : https://www.linkedin.com/in/tedwennmark/
MySQL 8.0
Upgrading to MySQL 8.0
1

Safe harbor statement
The following is intended to outline our general product direction. It is intended for information
purposes only, and may not be incorporated into any contract. It is not a commitment to deliver
any material, code, or functionality, and should not be relied upon in making purchasing
decisions.
The development, release, timing, and pricing of any features or functionality described for
Oracle’s products may change and remains at the sole discretion of Oracle Corporation.
2

• Long time MySQL user
– Started developing apps using MySQL over 20 years back
– Worked as MySQL DBA, Trainer and consultant the past
• MySQL Prinicipal Solution Architect at Oracle
• Work with normal MySQL but have focuse on distributed
databases with NDB Cluster.
• My workshops at Github: https://github.com/wwwted/
• Let’s stay in touch:
– https://www.linkedin.com/in/tedwennmark/
• Join us on slack: https://lefred.be/mysql-community-on-slack/
Ted Wennmark
3

MySQL 8.0 News
Program agenda
1
2 Upgrade to MySQL 8.0
4

MySQL is the most popular open-source database!
DB-Engines 2020
Database Ranking
5

MySQL is the most popular database!
Stack Overflow Developer Survey 2020
Developer Survey Results
6

- 3x Better Performance
- Replication Enhancements
- Optimizer Cost Model
- JSON Support
- Improved Security
- Sys & Performance Schema
- GIS
MySQL 5.7
MySQL HA Cluster
- MySQL Group Replication
- MySQL ReplicaSet
- MySQL Router
- MySQL Shell
MySQL 8.0
- 2x Better Performance
- NoSQL Document Store
- JSON
- CTEs
- Window Functions
- Data Dictionary
- InnoDB
- Roles
- Unicode
- GIS
2 Years in Development
400+ Worklogs
5000+ Bugs Fixed
500 New Tests
GA
MySQL Innovation: 5.7 -> 8.0
7

MySQL 8: What’s new!
DBA’s
 Transactional (InnoDB) Data Dictionary
 Persistent runtime configuration changes (SET PERSIST)
 Auto-increment counter value will now persist across server restarts!
 Indexing of PS tables
 SQL Roles (collection of privileges)
Developers
 Common Table Expressions
 Windows Functions
 Avoid locking with: SELECT ... FOR UPDATE [SKIP LOCKED | NOWAIT]
 More JSON and GIS functionality
8

• New! Support for the latest Unicode 9.0
• Per-country collation rules
– Accent Sensitive
– Case Sensitive
• UTF8MB4 as the default character set
– Project started in MySQL 5.7
– Many improvements to reduce performance impact
– The new default character set/collation: utf8mb4/utf8mb4_0900_ai_ci
MySQL 8: UTF8
9

MySQL 8: New Data Dictionary
• Metadata stored in InnoDB
– From files to a proven transactional storage engine
– System tables moved from MyISAM to InnoDB
• One source of truth
– One common data dictionary
• Atomic, crash safe DDL
– CREATE/DROP USER <u1, u2, u3>, DROP TABLE <t1, t2, t3>,...
10

MySQL 8: Alter Table - Instant Add Column
• Contribution from Tencent
– Only changes metadata
– No copying of data
– Smaller final data size
• ALTER TABLE … ADD COLUMN c, ALGORITHM = INSTANT (INPLACE, COPY)
• Supports DYNAMIC/COMPACT/REDUNDANT row formats
• Enabled by default
11

Feature Request
from DBAs
MySQL 8: Roles
• Improving MySQL Access Controls
• Easier to manage user and applications privileges
• Users can be assigned multiple roles
• SET DEFAULT ROLE to users or use SET ROLE during runtime
• Mandatory roles for all users can be configured on a server level
• Can export the role graph in GraphML
– SELECT ROLES_GRAPHML() into outfile '/tmp/test_roles.graphml';
– https://lefred.be/content/mysql-8-0-roles-and-graphml/
12

http://lefred.be/content/how-to-safely-upgrade-to-mysql-8-0/
MySQL 8 Shell: Upgrade Checker
Feature Request
from DBAs
• Quick and Easy MySQL Shell Utility
– JavaScript or Python
– MySQL 5.7 and onwards
– New util class
mysqlsh> util.checkForServerUpgade();
• Identifies potential issues for upgrade
• Recommends Fixes:
– Schema, configuration, tables, reserved keywords, use of
deprecated or removed features or datatypes and much
more
13

Performance
2x Faster than MySQL 5.7 for RO and RW @scale
• Scaling Read/Write Loads
– Re-designing how InnoDB writes to Redo log
• Utilizing IO Capacity
– Removing file system mutex
• High Contention Loads
– Contention Aware Transaction Scheduling
– CATS over FIFO
• Resource Groups
– Thread–CPU mapping can be managed
• UTF8MB4
• Partial JSON/BLOB update
• Information Schema
• Performance Schema
• Cost Model (mem/disk aware)
14

Production Workload #1
Production Workload #2
4x Improvement
on single-threaded!
MySQL 8: Parallel Replication
http://jfg-mysql.blogspot.se/2018/01/an-update-on-write-set-parallel-replication-bug-fix-in-mysql-8-0.html
• New! Improved Replication Throughput
• New! Improved Parallelism on slave,
independent of Master “load”
• Write-set extraction from Group
Replication drives multi-threaded
parallel slave performance more than
ever
• No more slave lag
16

MySQL GIS: Spatial Reference Systems (SRS) Support
”Matching or Exceeding PostgreSQL GIS Feature Set”
• MySQL 5.7
– The world is flat (SRS ID = 0)
– The word is infinite
– Axes are unitless
– Axes are orthogonal
– Axis order is irrelevant
– Axis direction is irrelevant
• MySQL 8.0
– The world can be flat or a ellipsoid
– Geographic coordinate system wrapes around
– Axes have units
– Geographic axes are not orthogonal
– Geographic axis order matter
– Axis direction my be relevant
17

GIS: Geography
• Earth’s surface, (longitude/latitude)
• Spatial Reference System (SRS)
– 5000 SRIDs from EPSG Geodetic Parameter Dataset
– SRID 4326 = WGS 84 («GPS coordinates») «Earth Elipsoid»
– INFORMATION_SCHEMA.ST_SPATIAL_REFERENCE_SYSTEMS
• SRID aware Spatial Datatypes
– CREATE TABLE t1 (g GEOMETRY SRID 4326);
– INFORMATION_SCHEMA.ST_GEOMETRY_COLUMNS
18

GIS: Geography (cont)
• SRID aware Spatial Indexes
– CREATE TABLE t1 (g GEOMETRY SRID 4326 NOT NULL, SPATIAL INDEX (g));
• SRID aware Spatial Functions
– ST_Distance(), ST_Length(),...
– ST_Within(), ST_Intersects(), ST_Contains(),...
• GeoHash(), GeoJSON()
19

MySQL 8.0 for ANALYTICS: CTEs & Window Functions
• New! Common Table Expression (WITH clause)
– Non-recursive
– Recursive
– Used for hierarchy traversal
• New! Window Functions
– Aggregation, ranking, analytics
– Used for analytics and reporting
20

MySQL 8.0: Invisible Indexes
• Indexes are “hidden” to the MySQL Optimizer
– Not the same as “disabled indexes”
– Contents are fully up to date and maintained by DML
• Use cases:
– Soft Delete (Recycle Bin)
– Staged Rollout
– Test
21

New Release Model: Continuous Delivery Model
• Starting with MySQL 8.0.11 we are changing our release model:
– Main driver is to release new features quicker
– More testing before releasing from our side
– New features disabled by default (if possible)
– No large re-factoring of code within CDM
• What does this means for minor upgrades:
– We are releasing new functionality and removing/deprecating unwanted features.
– Read our relase notes, pay extra attention to sections ” Deprecation and Removal Notes” and ”
Functionality Added or Changed”
– Treat minor upgrades ”almost” as a Major upgrade!!
22

New Release Model: Continuous Delivery Model ”My favorites”
• MySQL 8.0.11 (2018-04-19)
– LOCK INSTANCE FOR BACKUP
– RESTART SQL Statement
• MySQL 8.0.12 (2018-07-27)
– ”ALTER TABLE algorithm=INSTANT”
• MySQL 8.0.13 (2018-10-12)
– Functional indexes
• MySQL 8.0.14 (2019-01-21)
– Consistent reads in InnoDB Cluster
23
• MySQL 8.0.19 (2020-01-13)
– InnoDB ReplicaSet
• MySQL 8.0.20 (2020-04-27)
– Binlog Compression
– Support for hash outer, anti and
semi join
• MySQL 8.0.21 (2019-10-14)
– Disabling/enabling the Innodb
redo log dynamically
– Improved default values for
InnoDB Cluster and tagging
• MySQL 8.0.16 (2019-04-25)
– SQL: CHECK Constraints
– mysql_upgrade deprecated
• MySQL 8.0.17 (2019-07-22)
– MySQL Clone plugin
– Encrypted binary logs
• MySQL 8.0.18 (2019-10-14)
– EXPLAIN ANALYZE
– HASH joins

New Release Model: Continuous Delivery Model ”All new features”
https://mysqlserverteam.com/the-mysql-8-0-12-maintenance-release-is-generally-available/
24

MySQL 8.0 News
Program agenda
1
2 Upgrade to MySQL 8.0
25

Why upgrade?
• Performance/Scalability improvements
• New features in MySQL 8
• Stay on fully supported release
• Security improvements
• Regulatory compliance
26

Performance/Scalability
• Newer MySQL version scales better:
– MySQL 5.1 scales to ~4 cores
– MySQL 5.6 scales to ~36 threads (cores)
– MySQL 5.7 scales to ~64 threads (32 Core-HT)
– MySQL 8.0 scales to ~100 threads (test on 48 Cores-HT intel Skylake)
• Improvements in optimizer
– Better query plans with MRR, BKA.
– More work in storage engine with ICP
– Cost model can be tuned in 5.7 and in MySQL 8 Optimizer Takes Data Buffering into Account
– HASH JOIN in MySQL 8.0.18
– Histograms in MySQL 8.0
https://www.mysql.com/why-mysql/benchmarks/27

Support Life Cycle
Release GA Date Premier Support
End
Extended Support
End
Sustaining Support
End
MySQL 5.0 Oct 2005 Dec 2011 N/A Indefinite
MySQL 5.1 Dec 2008 Dec 2013 N/A Indefinite
MySQL 5.5 Dec 2010 Dec 2015 Dec 2019 Indefinite
MySQL 5.6 Feb 2013 Feb 2019 Feb 2021 Indefinite
MySQL 5.7 Oct 2015 Oct 2020 Oct 2023 Indefinite
MySQL 8.0 Apr 2018 Apr 2023 Apr 2026 Indefinite
http://www.oracle.com/us/support/library/lifetime-support-technology-069183.pdf
28

MySQL Security
• OpenSSL default in MySQL 8.0
• SQL ROLES added in MySQL 8.0.
• SSL enabled by default.
• Stronger passwords and validation of password quality.
• MySQL Enterprise Edition:
– MySQL Audit
– MySQL Monitor/Backup
– MySQL Firewall
– MySQL Encryption/TDE
– MySQL Data Masking
29

 Workbench
•Model
•Data
•Audit Data
•User Management
  Enterprise Monitor
•Identifies Vulnerabilities
•Security hardening policies
•Monitoring & Alerting
•User Monitoring
•Password Monitoring
•Schema Change Monitoring
•Backup Monitoring
Data Encryption
•TDE
•Encryption
•PKI
 Firewall
 Enterprise Authentication
•SSO - LDAP, AD, PAM
 Network Encryption
 Enterprise Audit
•Powerful Rules Engine
 Audit Vault
 Strong Authentication
 Access Controls
 Assess
 Prevent
 Detect
 Recover
 Enterprise Backup
•Encrypted
 HA
•Innodb Cluster
Thread Pool
•Attack minimization
 Key Vault
•Protect Keys
 Enterprise
Masking & De-Identification
•Masking
•Substitute/Subset
•Random Formatted Data
•Blacklisted Data
MySQL EE Security
30

MySQL Upgrade – Supported Paths
• All minor upgrades like 8.0.11 to 8.0.12 or doing a jump like 8.0.11 to 8.0.13
is supported.
• Upgrading one major (5.7 - 8.0) release level is supported. It’s
recommended upgrading to the latest 5.7 release before moving to 8.0.
• Doing long-jumps (5.6 – 8.0) is supported if you upgrade one release
level at a time.
• Direct upgrades that skip a release level (for example, upgrading directly
from MySQL 5.6 to 8.0) is not recommended nor supported.
31

MySQL Upgrade – Before you start I
• Review the “Release Notes” for target MySQL version, especially sections:
“Deprecation and Removal Notes”
“Functionality Added or Changed”
• Review the manual page on upgrading, take special care of sections covering:
“Changes in MySQL 8.0”
“Preparing Your Installation for Upgrade”
• Create a PLAN for your upgrade/downgrade.
• Create a BACKUP before starting upgrade.
http://dev.mysql.com/doc/refman/8.0/en/upgrading.html
https://mysqlserverteam.com/upgrading-to-mysql-8-0-here-is-what-you-need-to-know/
32

MySQL Upgrade – Before you start II
• Using MySQL Shell 8.0 we can check the status of the 5.7 server to be upgraded.
• Run MySQL Shell before upgrade:
– util.checkForServerUpgrade(“user@127.0.0.1:3306”, {"password":”secret", "targetVersion":"8.0.19",
"configPath":"/etc/my.cnf"})
– mysqlsh -- util check-for-server-upgrade user:password@127.0.0.1:3306 --target-version=8.0.19 --output-
format=JSON --config-path=/etc/my.cnf
• MySQL Shell will identify if you are using any reserved keywords, removed datatypes or
if some parameters in your configuration have new defaults and much more.
• Upgrade checker will not identify problems like GIS changes or deprecated MySQL
functions used by your application.
33

MySQL Upgrade – Remember!
• Performance/Scalability monitoring and testing (before and after).
• Advanced features (sp, functions or views).
• Do you need to consider replication between different versions of MySQL.
• Changes in default values between versions (sql_mode).
• Look out for statements creating warnings/errors (before upgrade).
• New optimizer features causing query plan changes.
• Monitor resource usage (iostat, top, vmstat etc).
35

MySQL Upgrade – In-place Upgrade
1. Do all necessary preparations/plans as explained earlier.
2. Do a graceful shutdown of MySQL (innodb_fast_shutdown=0)
Also look out for any existing XA transactions before upgrade.
3. Do upgrade, replace the old binaries with the new ones.
4. Start MySQL Server using new binaries.
5. Run mysql_upgrade binary to upgrade internal mysql repository.
(as of MySQL 8.0.16 this not needed)
6. Verify that all is working as expected!
https://dev.mysql.com/doc/refman/8.0/en/upgrade-binary-package.html#upgrade-procedure-inplace
36

MySQL Upgrade – Logical Upgrade
1. Do all necessary preparations/plans as explained earlier.
2. Do a dump of only the user databases/schemas (skipping the mysql
system schema, remember stored procedures, views and functions)
3. Do a clean installation of new MySQL version (remember to migrate
important my.cnf options and your users (mysqlpump)).
4. Load the dump files into the new MySQL server instance.
5. Verify that all is working as expected!
https://dev.mysql.com/doc/refman/8.0/en/upgrade-binary-package.html#upgrade-procedure-logical
http://mysqlserverteam.com/upgrading-directly-from-mysql-5-0-to-5-7-with-mysqldump/
37

MySQL Upgrade – Using Replication to Upgrade with minimal downtime
1. Do all necessary preparations/plans as explained
earlier.
2. Restore backup from production (5.7) database to
new replica server (8.0).
3. Start replication from source (5.7) to replica (8.0).
4. Once replica server has caught up with source.
5. Run some tests on replica (8.0)
6. Change application to connect to replica (8.0).
MySQL 5.7
MySQL
8.0
Application
Replication
https://dev.mysql.com/doc/refman/8.0/en/replication.html
38

MySQL Upgrade – Solving Regression Problems
• Explain is your friend!
– Optimizer_switch can help get back old query plans for slow queries.
• Monitor resources/MySQL before/after to spot changes.
– Without proper monitoring you have little chance to find root cause of problem.
• Did you change more than just MySQL version?
– Normally we see people refresh HW/OS at the same time which creates more sources for regressions.
• Use PERFORMANCE_SCHEMA and SYS schema to spot problems.
40

Upgrading from older MySQL versions to MySQL 8.0?
Important changes in MySQL 5.6
• Need at least version 5.1.20 of Connector/J (due to retire of “SET OPTION”).
• InnoDB “Files per table” now default.
• The Query Cache is disabled by default.
• 8 new reserved key words have been defined (“get” and “partition”).
• New data format of TIMESTAMP and DATETIME (microseconds).
• Password on command line causes warnings (problematic for scripts).
• New optimizer features may cause different query plans.
• Old password are not allowed by default.
41

Upgrading from older MySQL versions to MySQL 8.0?
• Old password format not allowed.
• innodb_strict_mode is “ON”.
• sql_mode option “NO_AUTO_CREATE_USER, STRICT_TRANS_TABLES,
ERROR_FOR_DIVISION_BY_ZERO, NO_ZERO_DATE, NO_ZERO_IN_DATE” is now
default.
• Changes to optimizer_switch, may cause changed query plans!
• Remove of IGNORE for ALTER TABLE and INSERT DELAYED.
• mysql_install_db script removed (now part of mysql --initialize).
• MySQL installation much more secure by default.
42

• Query Cache was removed.
• No support for non-native partitioning (no partitioning for MyISAM).
• No foreign key constraint names longer than 64 characters.
• Removal of some combined sql_mode definitions.
• More reserved keywords (groups, member, admin, system) (Application impact).
• New default character set and collation (utf8mb4) (Application impact).
• New default authentication plug-in (Application impact).
• Binary log enabled by default (log_bin = ON).
• GIS have changed (Application impact).
43

New release model with MySQL 8.0: Continuous Delivery Model
Minor upgrades
• Important to treat MySQL 8.0 minor upgrades the same way as major upgrades.
• MySQL 8.0 minor releases can change default values and add, changes or deprecate
functionality.
• Follow best practices by reading release notes for minor upgrades, especially sections:
“Deprecation and Removal Notes”
“Functionality Added or Changed”
• Use MySQL Shell Upgrade checker to spot upgrade problems.
• Take backup before and after upgrade of minor releases.
• Downgrade to previous minor release is only supported via restore from backup.
“Cannot boot server version 80016 on data directory built by version 80017. Downgrade is not supported”
• Use matching versions of backup software (MySQL 8.0.19 and MEB 8.0.19).
44

Why MySQL Enterprise Edition?
Insure Your Deployments
Get the Best Results
Delight Customers
Improve
Performance &
Scalability
Enhance Agility &
Productivity
Reduce TCO
Mitigate
Risks
Get
Immediate
Help if/when
Needed
Increase
Customer
Satisfaction
46

Management
Tools
Advanced Features Support
•Scalability
•High Availability
•Security
•Encryption + TDE
•Data Masking
•Firewall
•Monitoring
•Backup
•Development
•Administration
•Migration
• Technical Support
• Consultative Support
• Oracle Certifications
MySQL Enterprise Edition
47

MySQL Enterprise Backup
• Online, non-locking backup and recovery
– Complete MySQL instance backup (data and config)
– Partial backup and restore
• Direct Cloud storage backups
– Oracle Storage Cloud, S3, etc.
• Incremental backups
• Point-in-time recovery
• Advanced compressed and encryption
• Backup to tape (SBT)
• Cross-Platform (Windows, Linux, Unix)
48

MySQL Enterprise Support
• Largest MySQL engineering and support organization
• Backed by the MySQL developers
• World-class support, in 29 languages
• Hot fixes & maintenance releases
• 24x7x365
• Unlimited incidents
• Consultative support
• Global scale and reach
Get immediate help for any MySQL
issue, plus expert advice
49

MySQL Enterprise Consultative Support
• Remote troubleshooting
• Replication review
• Partitioning review
• Schema review
• Query review
• Performance tuning
• ...and more
Get immediate help for any MySQL
issue, plus expert advice
50

More information
• http://dev.mysql.com/doc/refman/8.0/en/upgrading.html
• https://mysqlserverteam.com/upgrading-to-mysql-8-0-here-is-what-you-need-to-know/
• https://mysqlserverteam.com/mysql-shell-8-0-4-introducing-upgrade-checker-utility/
• http://lefred.be/content/how-to-safely-upgrade-to-mysql-8-0/
• http://lefred.be/content/migrating-to-mysql-8-0-without-breaking-old-application/
• https://saveriomiroddi.github.io/Pre-fosdem-talk-upgrading-from-mysql-5.7-to-8.0/
• https://dev.mysql.com/doc/refman/8.0/en/
51

Thank you
Ted Wennmark
MySQL Solution Architect EMEA
MySQL Business Unit
52

01 upgrade to my sql8

Recommended

More Related Content

What's hot (20)

Similar to 01 upgrade to my sql8 (20)

More from Ted Wennmark (11)

Recently uploaded (20)

01 upgrade to my sql8