MySQL High Availability Solutions - Avoid loss of service by reducing the risk of failures

MySQL HA Solutions
© 2021 Oracle MySQL
Olivier Dasini
MySQL Principal Solutions Architect EMEA
olivier.dasini@oracle.com
Twitter : @freshdaz
Blogs : http://dasini.net/blog/en
: http://dasini.net/blog/fr
Avoid loss of service by reducing the risk of failures

Safe harbor statement
The following is intended to outline our general product direction. It is intended for information
purposes only, and may not be incorporated into any contract. It is not a commitment to deliver
any material, code, or functionality, and should not be relied upon in making purchasing
decisions.
The development, release, timing, and pricing of any features or functionality described for
Oracle’s products may change and remains at the sole discretion of Oracle Corporation.
2 © 2021 Oracle MySQL

3
Me, Myself & I
3
Olivier DASINI
Copyright © 2021 Oracle and/or its affiliates. All rights reserved.

MySQL Geek
 Addicted to MySQL for 15+ years
 Playing with databases for 20+ years

MySQL Writer, Blogger and Speaker
 Also former : DBA, Consultant, Architect, Trainer, ...

MySQL Principal Solutions Architect EMEA at Oracle

Stay up to date!
 Twitter: @freshdaz
 Blog: www.dasini.net/blog/en

Why MySQL ?
Program agenda
1
2
3
4
High Availability
MySQL InnoDB Cluster w/ MySQL Group Replication
MySQL InnoDB ReplicaSet w/ MySQL Replication
MySQL NDB Cluster
5

Why MySQL ?

The world’s most popular open source database

Some Of Our Customers
 MySQL powers the most demanding Web, E-commerce, SaaS and Online Transaction Processing
(OLTP) applications
 It is a fully integrated transaction-safe, ACID compliant database with full commit, rollback, crash
recovery and row level locking capabilities
 MySQL delivers the ease of use, scalability, and performance to power:

Explosive Data Growth
Source IDC, 2017 Source Domo, 2018

Title only
They Scale
with
MySQL
Mobile Network
Supporting Over 800
Million Subscribers
2+ Billion Active Users
100 TB of User Data for PayPal
IDs Processed for
1 Billion Citizens
850 Million Candy
Crush Game Plays/Day
2 Billion Events/Day for
Booking.com

High Availability

High Availability: Factors
 Environment
– Redundant servers in different datacenters and geographical areas will protect you against regional issues—
power grid failures, hurricanes, earthquakes, etc.
 Hardware
– Each part of your hardware stack—networking, storage, servers—should be redundant
 Software
– Every layer of the software stack needs to be duplicated and distributed across separate hardware and
environments
 Data
– Data loss and inconsistency/corruption must be prevented by having multiple copies of each piece of data,
with consistency checks and guarantees for each change

High Availability: Database Needs
 Management infrastructure
– Monitoring of status, health, performance
●
Facilities for service changes, service transitions
 Failure detection and handling
– Identify and handle failures
 Elasticity
– Scale up to ensure acceptable performance is always maintained
 Consistency guarantees
– Conflict detection and handling ; data loss protection/prevention
 Online maintenance

High Availability: Causes of Downtime
A study by the Gartner Group projected that
through 2015, 80% of downtime will be due to
people and process issues
40%
40%
20%
Software/Application Human Error
Hardware
* Source: Gartner Group 1998 survey

High Availability: The Business Cost of Downtime
 Calculate a cost per minute of downtime
– Average revenue generated per-minute over a year
– Cost of not meeting any customer SLAs
– Factor in costs that are harder to quantify
1. Revenue
2. Reputation
3. Customer sentiment
4. Stock price
5. Service’s success
6. Company’s very existence
THIS is why
HA matters!

Database High Availability

Multiple copies of data with replication process

MySQL Replication modes for High Availability
 Asynchronous => MySQL InnoDB ReplicaSet powered by MySQL Replication
– MySQL Default; In parallel: Source ACKs to app and sends transaction to Replica
– Fast; Risk of lost changes if Source dies
 Semi-Synchronous => MySQL Replication with semi-synchronous plugin
– MySQL 5.5+ Enhanced in MySQL 5.7; Serially: Source waits for change to be received by Replica then In parallel ACK
to app and apply changes on Replica
– Intermediate latency; Lossless (MySQL 5.7)
 Virtual-Synchronous => MySQL InnoDB Cluster powered by MySQL Group Replication
– MySQL Group Replication / MySQL InnoDB Cluster; Multi-Sources updates on nodes applied in parallel
– MySQL Plugin delivered by Oracle MySQL for MySQL 5.7+
– Intermediate latency; Best suited to small transactions; Lossless
 Synchronous => MySQL NDB Cluster
– Only available with MySQL NDB Cluster; Serially: Source waits for change to be applied on all Replicas before ACK to
app
– Higher latency; If Active/Active, best suited to small transactions; Lossless

MySQL InnoDB Cluster
Collection of products that work together to provide a complete High Availability solution for MySQL

“High Availability becomes a
core first class feature of MySQL!”
App Servers with
MySQL Router
MySQL Shell
Setup, Manage,
Orchestrate
App Clients
MySQL Group Replication

MySQL InnoDB Cluster - Goals
Flexible and Modern
– SQL & NoSQL together
– Protocol Buffers
– Asynchronous API
– Developer friendly
A single product: MySQL
– All components created together
– Tested together
– Packaged together
Easy to use
– One client: MySQL Shell
– Easy packaging
– Integrate orchestration
– Homogenous servers
Completely Open Source - GPL
– No license required to have
– High Availability

MySQL InnoDB Cluster – The Big Picture
P
App
S S
MySQL Shell and Orchestration Tooling
App
Simple Mapping,
State and Extra
Metadata
Control, Coordinate,
Provision
Monitoring (MEM)
MySQL Router
MySQL Router

Native, built-in High Availability for your MySQL databases

App Servers with
MySQL Router
MySQL Shell
Setup, Manage,
Orchestrate
App Clients

MySQL Group Replication – What Is It ?
 Group Replication library
– Implementation of Replicated Database State Machine theory
●
MySQL GCS is based on Paxos (variant of Mencius)
– Provides virtually synchronous replication for MySQL 5.7+
– Supported on all MySQL platforms
●
Linux, Windows, Solaris, OSX, FreeBSD
– Open source
●
GPL
“Single/Multi-Source update everywhere replication plugin for MySQL with built-in
automatic distributed recovery, conflict detection & group membership.”
http://dasini.net/blog/2016/11/08/deployer-un-cluster-mysql-group-replication/

MySQL Group Replication – What Does It Provide ?
 A Highly Available distributed MySQL database service
– Clustering eliminates single points of failure (No SPOF)
●
Allows for online maintenance
– Removes the need for handling server fail-over
– Provides fault tolerance
– Enables update everywhere setups
– Automates group reconfiguration (handling of crashes, failures, re-connects)
– Provides a highly available replicated database
– Automatically ensures data consistency
●
Detects and handles conflicts
●
Prevents data loss
●
Prevents data corruption

MySQL Group Replication – Architecture
Node Types
R: MySQL Router (or any proxy like ProxySQL, HAProxy...)
M: mysqld nodes participating in MySQL Group Replication

Single Primary Mode
 Configuration mode that makes a single member act as a writeable Source (PRIMARY) and the rest of the
members act as hot-standbys (SECONDARY)
– The group itself coordinates automatically to figure out which is the member that will act as the PRIMARY, through an
automatic primary election mechanism
– Secondaries are automatically set to read-only
 Single primary mode is the default mode
– Closer to classic asynchronous replication setups, simpler to reason about the beginning
– Avoids some limitations of multi-primary mode by default
 You can describe the status of the cluster by executing status() function :
MySQL nodeA JS> cluster.status()
{
"clusterName": "test",
"defaultReplicaSet": {
"name": "default",
"primary": "nodeA:3306",
"ssl": "REQUIRED",
"status": "OK",
"statusText": "Cluster is ONLINE and can tolerate up to ONE failure.",
"topology": {
...
MySQL nodeA JS> cluster.status()
{
"clusterName": "test",
"defaultReplicaSet": {
"name": "default",
"primary": "nodeA:3306",
"ssl": "REQUIRED",
"status": "OK",
"statusText": "Cluster is ONLINE and can tolerate up to ONE failure.",
"topology": {
...

Control Primary Election - Choose Your Primary
MySQL nodeB JS> cluster.setPrimaryInstance("nodeA:3306")
MySQL nodeB JS> cluster.setPrimaryInstance("nodeA:3306")
User tells current primary to give up its role and assign it to another server

Control Primary Promotion - Priorities
MySQL nodeA JS> cluster.setInstanceOption("nodeB:3306", "memberWeight", 100)
MySQL nodeA JS> cluster.setInstanceOption("nodeC:3306", "memberWeight", 90)
MySQL nodeA JS> cluster.setInstanceOption("nodeB:3306", "memberWeight", 100)
MySQL nodeA JS> cluster.setInstanceOption("nodeC:3306", "memberWeight", 90)
Choose next primary by assigning election weights to the candidates

Multi Primary Mode
 Configuration mode that makes all members writeable
– Enabled by running <cluster>.switchToMultiPrimaryMode() function
 Any two transactions on different servers can write to the same tuple
 Conflicts will be detected and dealt with
– First committer wins rule

Consistency Levels
 EVENTUAL
EVENTUAL (default, current behavior)
– Transaction does not wait at all
– Executes on the current snapshot of the data on that member
 BEFORE
BEFORE (Synchronize on Reads)
– Transaction waits for all transactions to complete
– Executes on the most up to date snapshot of the data in the group
 AFTER
AFTER (Synchronize on Writes)
– Transaction waits until all members have executes if
– Executes on the current snapshot of the data on that member
group_replication_consistency: Control the transaction consistency guarantee
https://dev.mysql.com/doc/refman/8.0/en/group-replication-options.html#sysvar_group_replication_consistency

Consistency Levels
 BEFORE_AND_AFTER
BEFORE_AND_AFTER (Yes you can combine both)
– Transaction waits for all preceding transactions and for all members to execute it
– Executes on the most up to date snapshot of the data in the group and updates everywhere before returning to the
application
 BEFORE_ON_PRIMARY_FAILOVER
BEFORE_ON_PRIMARY_FAILOVER (fencing mechanism used by new elected primary in single primary)
– Transaction waits for all transactions in the new primary’s replication backlog to be executed
– Executes on the snapshot of the data that the old primary was in when it stepped down (or crashed)
group_replication_consistency: Control the transaction consistency guarantee
https://dev.mysql.com/doc/refman/8.0/en/group-replication-options.html#sysvar_group_replication_consistency

Fast Group Replication Recovery: Clone Support
Empty or delayed replica quickly online by using Automatic Cloning and WRITESET
Requires that the CLONE plugin is installed. The CLONE plugin is shipped with MySQL 8.0.17+

Other Operations features
 Preventing (involuntary) writes on stale members
– Automatically setting server to Read-Only
 Preventing (involuntary) Reads/Writes on stale members
– Automatically shoot member in the Head (ST*NITH)
 Control Primary Election: Choose Your Primary
– User tells current primary to give up its role and assign it to another server
 Dealing With Unreliable Networks: Relaxed Member Eviction
– User controls the amount of time to wait until others decide to evict a member from the group
 Automatic Cluster-Rejoin on Partitions
– Member tries to rejoin automatically in case it gets evicted

Full stack secure connections
 Following the industry standards, Group Replication supports secure connections along the
complete stack
– Client connections
– Distributed recovery connections
– Connections between members
 IP Allowlisting
– Restrict which hosts are allowed to connect to the group
– By default it is set to the values AUTOMATIC, which allow connections from private subnetworks active on the
host
http://mysqlhighavailability.com/mysql-group-replication-securing-the-perimeter/

Binary Log Encryption on Disk
 Protects binary log data at rest
 Controllable using a system variable: binlog_encryption
 Two tier encryption, one master key and one key per file
https://mysqlhighavailability.com/binary-log-encryption-at-rest/

MySQL Group Replication – Main Requirements
 InnoDB Storage Engine
– Data must be stored in the InnoDB transactional storage engine
 Primary Keys
– Every table that is to be replicated by the group must have an explicit primary key defined
 Network Performance
– Group Replication is designed to be deployed in a cluster environment where server instances are very close to
each other, and is impacted by both network latency as well as network bandwidth

MySQL Group Replication - Wrap-Up
Storage Engine InnoDB
Distributed Architecture Shared nothing
Consistency Model Eventual to Strong consistency (tunable)
NoSQL APIs MySQL Document Store
Replication mode Virtually Synchronous (Paxos)
Database Failover Automatic
OS Supported Linux, Windows, Solaris, OSX, FreeBSD
Operational Complexity Easy! Thanks to MySQL InnoDB Cluster
Administration Standard (MySQL)

MySQL Shell
App Servers with
MySQL Router
MySQL Shell
Setup, Manage,
Orchestrate
App Clients
”MySQL Shell provides the developer
and DBA with a single intuitive, flexible,
and powerful interface for all MySQL
related tasks!”

MySQL Shell Overview
MySQL
Server 5.7
MySQL 8.0
Upgrade Checker
Prompt Themes
Auto Completion
&
Command History
MySQL
Server 8.0
Document
StoAre
X Dev API
SQL CLI
Output Formats
(Table, JSON,
Tabbed)
Batch Execution
JavaScript
Python
SQL
importJSON
ParalleI
importTABLE
InnoDB Cluster/
ReplicaSet

MySQL Shell - Key Features
 Scripting for Javascript, Python, and SQL mode
 Supports MySQL Standard and X Protocols
 Document and Relational Models
 CRUD Document and Relational APIs via scripting
 Traditional Table, JSON, Tab Separated output results formats
 Both Interactive and Batch operations
 Open Source - GPL
Interface for Development and Administration of MySQL

MySQL Shell – Admin API
 Creation and Management of InnoDB Clusters
– Hide the complexity of Configuration, Provisioning, Orchestration
– JS> dba.help()
 The global variable 'dba' is used to access the MySQL AdminAPI
 Perform DBA operations
– Manage MySQL InnoDB Cluster (& MySQL ReplicaSet)
●
Create clusters
●
Get cluster info
●
Start/Stop MySQL Instances
●
Validate MySQL instances
●
…
Database Administration Interface

Deploy an InnoDB Cluster
1/ Connect to the first node
$ mysqlsh admin@192.168.1.1:3306
2/ Create the cluster
JS> var cluster=dba.createCluster('demoCluster');
3/ Add the second node
JS> cluster.addInstance('root@192.168.1.2:3306')
4/ Add the third node
JS> cluster.addInstance('root@192.168.1.3:3306')
5/ Check the InnoDB cluster status
JS> cluster.status()

MySQL Router
App Servers with
MySQL Router
MySQL Shell
Setup, Manage,
Orchestrate
App Clients

MySQL Router
 MySQL Router is lightweight middleware that provides transparent routing between your
application and back-end MySQL Servers
– Load balancing
– Application connection failover
 MySQL Router also provides High Availability and Scalability by effectively routing database traffic
to appropriate back-end MySQL Servers
 The pluggable architecture also enables developers to extend MySQL Router for custom use cases
Lightweight middleware that provides transparent routing

MySQL Router
 Native support for MySQL InnoDB Cluster (& MySQL ReplicaSet)
– Understands Group Replication topology
– Utilizes metadata schema stored on each member
●
Bootstraps itself and sets up client routing for the GR cluster
●
Allows for intelligent client routing into the GR cluster
●
Supports multi-source and single primary modes
 Core improvements
– Logging
– Monitoring
– Performance
– Security

MySQL InnoDB Cluster - Wrap-Up
App Servers with
MySQL Router
MySQL Shell
Setup, Manage,
Orchestrate
App Clients

MySQL InnoDB ReplicaSet
Administer a set of MySQL instances running asynchronous replication

“A quick & easy way to get
MySQL Replication & MySQL Router
up and running”
App Servers with
MySQL Router
MySQL Replication
MySQL Shell
Setup, Manage,
Orchestrate
App Clients

MySQL InnoDB ReplicaSet - Introduction
 An InnoDB ReplicaSet consists of a single source & multiple replicas (primary/secondary)
 Enable you to administer a set of MySQL instances running asynchronous GTID-based replication in a
similar way to InnoDB Cluster
 Enable you to work with MySQL replication without a deep understanding of the underlying concepts
 It’s a quick and easy way to get MySQL Replication and MySQL Router up and running
 You can automatically configure MySQL Router to use your InnoDB ReplicaSet
 Making it well suited to:
– Scaling out reads
– Provide manual failover capabilities in use cases that do not require the high availability offered by InnoDB Cluster

MySQL InnoDB ReplicaSet - Goals
Flexible and Modern
– SQL & NoSQL together
– Protocol Buffers
– Asynchronous API
– Developer friendly
A single product: MySQL
– All components created together
– Tested together
– Packaged together
– Proven & well known async replication
Easy to use
– One client: MySQL Shell
– Easy packaging
– Integrate orchestration
– Homogenous servers
Completely Open Source - GPL
– No license required to have
– High Availability

MySQL InnoDB ReplicaSet – The Big Picture
S
App
R
R
MySQL Shell and Orchestration Tooling
App
Simple Mapping,
State and Extra
Metadata
Control,
Coordinate,
Provision
Monitoring (MEM)
MySQL Router
MySQL Async Replication
MySQL Router

MySQL InnoDB ReplicaSet - Requirements & Limitations
 Requirements:
– MySQL 8.0.19+
– GTID-based & Row based replication
 Limitations:
– Manual failover -- which is a good thing
●
No coordination between servers mandates external arbitration
●
However, secondary instances remain available for reads.
– No multi-primary as such topology cannot guarantee data consistency
●
No data reconciliation
●
No conflict handling
●
No protection from partial data loss
– All secondary members replicate from primary
●
Single tiered replication support.
 It is recommended that you deploy InnoDB Cluster wherever possible
https://dev.mysql.com/doc/refman/8.0/en/introducing-innodb-replicaset.html

MySQL Replication
Enable data from one source to be copied to one or more replicas
https://dev.mysql.com/doc/refman/8.0/en/replication.html

MySQL Replication
App Servers with
MySQL Router
MySQL Shell
Setup, Manage,
Orchestrate
App Clients
MySQL Replication

MySQL Replication Overview
Sender
thread
Receiver
thread
Applier
Threads
SOURCE REPLICA

Binary Log
 Logical replication log recording Source changes (binary log)
 Row or statement based format (may be intermixed)
 Each transaction is split into groups of events
 Control events: Rotate, Format Description, GTID, and mode...

Global Transation Identifiers (GTID) Based MySQL Replication
 GTID is simple to track & compare replication across MySQL instances
– Unique identifier for each transaction written to the Binlog
●
source_id:transaction_id
●
eg : 3E11FA47-71CA-11E1-9E33-C80AA9429562:12
 Automatically identify the most up-to-date Replica for failover
 Simple to determine whether sources and replicas are consistent
 When using GTIDs you can take advantage of:
– Auto-positioning
– Easy fail-over,
– Monitor replicated transactions using Performance Schema tables
Self Healing MySQL Replication
Source
GTID=AAA:123
GTID=AAA:123
GTID=AAA:123
Replica 1
Replica 2
https://dev.mysql.com/doc/refman/8.0/en/replication-gtids.html

Background: What is Replication Used For?
S R
R2
R1
Rn
S
Write clients Read clients
Read clients
Write clients
More
reads?
More
Replicas!
Read scale-out

S
R
Write clients
Business intelligent client applications
Reporting client applications
Big queries client applications
On-line Backup and Reporting

C
B
B
A
A
C
Image from
www.ginkgomaps.com

Background: Use MySQL Replication For HA
B
C
Crash
Crash
B is the
new Source
Ouch!!! Whew!
B
C
A
C
B
Redundancy: If Source crashes, promote a Replica as the new Source

MySQL Replication - Wrap-Up
Storage Engine All (InnoDB only if using InnoDB ReplicaSet)
Consistency Model Eventual consistency
NoSQL APIs MySQL Document Store
Replication mode Asynchronous & Semi-synchronous
Database Failover Manual
Operational Complexity Very Easy!
Administration Standard (MySQL)

MySQL InnoDB ReplicaSet Features
https://dev.mysql.com/doc/refman/8.0/en/mysql-innodb-replicaset.html

Fast ReplicaSet provisioning: Clone Support
Empty or delayed replica quickly online by using Automatic Cloning and WRITESET
Requires that the CLONE plugin is installed. The CLONE plugin is shipped with MySQL 8.0.17+

Planned Changes of the ReplicaSet Primary
MySQL Router JS> rs.setPrimaryInstance("nodeA:3306")
MySQL Router JS> rs.setPrimaryInstance("nodeA:3306")
User tells current primary to give up its role and assign it to another server

Full stack secure connections
 Following the industry standards, MySQL ReplicaSet supports secure connections along the
complete stack
– Client connections
– Distributed recovery connections
– Connections between members

Binary Log Encryption on Disk
 Protects binary log data at rest
 Controllable using a system variable: binlog_encryption
 Two tier encryption, one master key and one key per file
https://mysqlhighavailability.com/binary-log-encryption-at-rest/

MySQL Shell
App Servers with
MySQL Router
MySQL Shell
Setup, Manage,
Orchestrate
App Clients
MySQL Replication
”MySQL Shell provides the developer
and DBA with a single intuitive,
flexible, and powerful interface for all
MySQL related tasks!”

MySQL Shell Overview
MySQL
Server 5.7
MySQL 8.0
Upgrade Checker
Prompt Themes
Auto Completion
&
Command History
MySQL
Server 8.0
Document
Store
X Dev API
SQL CLI
Output Formats
(Table, JSON,
Tabbed)
Batch Execution
JavaScript
Python
SQL
importJSON
ParalleI
importTABLE
InnoDB Cluster/
ReplicaSet

MySQL Shell - Key Features
 Scripting for Javascript, Python, and SQL mode
 Supports MySQL Standard and X Protocols
 Document and Relational Models
 CRUD Document and Relational APIs via scripting
 Traditional Table, JSON, Tab Separated output results formats
 Both Interactive and Batch operations
Interface for Development and Administration of MySQL

MySQL Shell – Admin API
 Creation and Management of InnoDB ReplicaSets & Clusters
– Hide the complexity of Configuration, Provisioning, Orchestration
– JS> dba.help()
 The global variable 'dba' is used to access the MySQL AdminAPI
 Perform DBA operations
– Manage MySQL InnoDB ReplicaSet & MySQL InnoDB Cluster
●
Configure MySQL instances
●
Create ReplicaSets/Clusters
●
Get ReplicaSet/Cluster info
●
Start/Stop MySQL Instances
●
…
Database Administration Interface

Deploy an InnoDB ReplicaSet
1/ Connect to the first node
$ mysqlsh admin@192.168.2.1:3306
2/ Create the replicaset
JS> var rs=dba.createReplicaSet('demoReplicaSet');
3/ Add a new node
JS> rs.addInstance('root@192.168.2.2:3306')
4/ Check the replicaset cluster status
JS> rs.status()

MySQL Router
App Servers with
MySQL Router
MySQL Shell
Setup, Manage,
Orchestrate
App Clients
MySQL Replication

MySQL Router
 MySQL Router is lightweight middleware that provides transparent routing between your application
and back-end MySQL Servers
– Load balancing
– Application connection failover
 MySQL Router also provides High Availability and Scalability by effectively routing database traffic to
appropriate back-end MySQL Servers
 The pluggable architecture also enables developers to extend MySQL Router for custom use cases

MySQL Router
 Native support for MySQL InnoDB ReplicaSet & MySQL InnoDB Cluster
– Understands Replication topology
– Utilizes metadata schema stored on each member
●
Bootstraps itself and sets up client routing for the ReplicaSet
●
Allows for intelligent client routing into the ReplicaSet
 Core improvements
– Logging
– Monitoring
– Performance
– Security

MySQL InnoDB ReplicaSet - Wrap-up
 Automatically provisioning new members: InnoDB Clone
 MySQL Shell Automatically conﬁgures users & Replication
 Integrated MySQL Router load balancing
 Automatic Router Bootstrapping – no config
 Router is stateless, adapts to topology changes
 Easy to use manual switchover/failover
 Shell conﬁgures Server, Router, Replication in a standardized best practice setup, prevents mistakes
 Standard Solution
 Supported & QA'ed by Oracle MySQL
 Easy to use, even for MySQL beginner
– Enable you to work with MySQL replication without a deep understanding of the underlying concepts

MySQL NDB Cluster

MySQL NDB Cluster
 MySQL NDB Cluster
– In memory distributed database
– NDB storage engine
– Automatic sharding
– Native access via several API
– Read/write consistency
– Read/write scalability
– Management via MCM/mgm-tool
Rock Solid
Availability
In-Memory
Real-Time
Performance
Extreme
Read & Write
Scalability
Elasticity

When TO consider MySQL NDB Cluster
 You need High Availability 6-9’s (and strong consistency)
 You need Sharding, either due to size or write performance
 You need Linear Scalabillity when adding more nodes
 You need predictable Real-time response times
– SQL and cross shard join support
 You want a ACID distributed in-memory database
81

MySQL NDB Cluster Architecture overview
MySQL NDB Cluster Data Nodes
Clients
Application Layer
Data Layer
Management

MySQL NDB Cluster Architecture
 Multiple data nodes form a cluster
 Shared nothing architecture
 Data is automatically distributed to data nodes
83

MySQL NDB Cluster Architecture - Replicas
Copy of 1
Copy of 1
 Multiple copies of data are maintained for availability
 A group of data nodes shares the same data
 1 - 4 replicas/copies of data can be configured
84

User-id (PK) Service Data
1773467253 chat xxx
6257346892 chat xxx
1773467253 photos xxx
7234782739 photos xxx
8235602099 reminders xxx
8437829249 location xxx
MySQL Cluster Data Nodes
Partition Key
Data distribution
 Auto-partitioning and distribution
– No name-node or central master
 Each dataset is split into fragments and
distributed across data nodes.
 Within a cluster data is always consistent.
85

Writing data “ACID”
Data
Memory
Flush writes to disk in
background checkpoints
Commit Log (REDO)
writes
…
time
…
 Data in MySQL Cluster is written to memory
and disk in a way that allows real-time
access and recovery
 Memory is locked so it won’t swap
 Writes go to data memory and commit log
 Background process checkpoints data
memory for recovery
 Reads always happen from memory -
not from disk
86

MySQL NDB Cluster Use Cases
 Key-Value store
+ High Availability, Scale-Out, Durability
 Transactional object store
+ Multi row transactions and consistency
 Relational database with MySQL Server front-ends
+ SQL joins, foreign keys, triggers, stored procedures, generated columns, JSON
87

MySQL NDB Cluster - Wrap-Up
Storage Engine NDB
Consistency Model Strong consistency
NoSQL APIs Native NDB API
Replication mode Synchronous (2PC)
Database Failover Automatic
Operational Complexity High (medium if using MCM)
Administration Custom (MySQL & NDB)

Monitoring

MySQL Enterprise Monitor
 See a central view of the complete MySQL env
 Real-time MySQL performance and availability monitoring
 Visually find & fix problem queries with the Query Analyzer
 Alerts through email or text message
 Disk monitoring for capacity planning
 Replications monitoring
 Role based access controls
 Cloud friendly architecture
– No agents required
– Policy driven configuration
– Easy integration with DevOps tools
A DBA’s assistant: Visualize problems and tuning opportunities.

M
MySQL E
Enterprise M
Monitor – Group Replication Support
 Native holistic support for Group Replication clusters
– Intelligent monitoring and alerting
– Topology views
– Detailed metrics and graphs
– Best Practice advice
 Monitoring of MySQL Group Replication
https://www.mysql.com/products/enterprise/monitor.html

M
MySQL E
Enterprise M
Monitor – Group Replication Support
 Group Replication with 3 online nodes
 Group Replication with 3 nodes :
– 2 onlines
– 1 unreachable

M
MySQL E
Enterprise M
Monitor
 Asynchronous replication between
– Group Replication cluster : Primary
– Standalone instances : Secondaries

M
MySQL E
Enterprise M
Monitor
 Asynchronous replication between
– Group Replication cluster 1 : Primary
– Group Replication cluster 2 : Secondary

M
MySQL E
Enterprise M
Monitor – MySQL Replication Support
 Native holistic support for MySQL Replication
– Intelligent monitoring and alerting
– Topology views
– Detailed metrics and graphs
– Best Practice advice
 Monitoring of MySQL Replication architectures
https://www.mysql.com/products/enterprise/monitor.html

M
MySQL E
Enterprise M
Topology

M
MySQL E
Enterprise M
Replication issue

M
MySQL E
Enterprise M
Statistics

M
MySQL E
Enterprise M
Monitor – NDB Cluster Support
 Auto discovery and grouping of NDB processes
– ndbapi
– ndb_mgmd
– ndbd/ndbmtd
 Visual cluster topology
 New reports and graphs
 New advisors
Visibility into the performance, availability, and health of the cluster

MySQL Enterprise Edition
10
0
© 2021 Oracle MySQL
Management Tools
Advanced Features Support
• Authentication
• Audit
• Encryption / TDE
• Firewall
• Data Masking
• High Availability
• Monitoring
• Backup
• Development
• Administration
• Migration
• Technical Support
• Consultative Support
• Oracle Certifications
Reduce the risk, complexity and cost

MySQL Security Achitecture

Wrap-up

“High Availability becomes a
core first class feature of MySQL!”
App Servers with
MySQL Router
MySQL Shell
Setup, Manage,
Orchestrate
App Clients
Collection of products that work together to provide a complete High Availability solution for MySQL

“A quick & easy way to get
MySQL Replication & MySQL Router
up and running”
App Servers with
MySQL Router
MySQL Replication
MySQL Shell
Setup, Manage,
Orchestrate
App Clients

MySQL NDB Cluster
 MySQL NDB Cluster
– In memory distributed database
– NDB storage engine
– Automatic sharding
– Native access via several API
– Read/write consistency
– Read/write scalability
– Management via MCM/mgm-tool
Rock Solid
Availability
In-Memory
Real-Time
Performance
Extreme
Read & Write
Scalability
Elasticity

Resources

Resources – MySQL InnoDB Cluster 1/2
 Tutoriel – Déployer MySQL 8.0 InnoDB Cluster
http://dasini.net/blog/2019/09/03/tutoriel-deployer-mysql-8-0-innodb-cluster-09-2019/
 MySQL InnoDB Cluster – Easy Recovering and provisioning
http://dasini.net/blog/2019/09/10/mysql-innodb-cluster-easy-recovering-and-provisioning/
 MySQL InnoDB Cluster – Recovering and provisioning with MySQL Enterprise Backup
http://dasini.net/blog/2019/07/11/mysql-innodb-cluster-recovering-and-provisioning-with-mysql-enterprise-backup/
 MySQL InnoDB Cluster Documentation
https://dev.mysql.com/doc/refman/8.0/en/mysql-innodb-cluster-introduction.html
 Using MySQL AdminAPI
https://dev.mysql.com/doc/mysql-shell/8.0/en/mysql-innodb-cluster.html
 MySQL Group Replication Documentation
https://dev.mysql.com/doc/refman/8.0/en/group-replication.html

Resources – MySQL InnoDB Cluster 2/2
 MySQL Enterprise Backup – User Guide
https://dev.mysql.com/doc/mysql-enterprise-backup/8.0/en/
 MEB – Using MySQL Enterprise Backup with Group Replication
https://dev.mysql.com/doc/mysql-enterprise-backup/8.0/en/meb-group-replication.html
 How to use MySQL Enterprise Backup to back up and subsequently restore a Group Replication
https://dev.mysql.com/doc/refman/8.0/en/group-replication-enterprise-backup.html
 MySQL InnoDB Cluster: is the router a single point of failure ?
https://lefred.be/content/mysql-innodb-cluster-is-the-router-a-single-point-of-failure/
 MySQL Router HA with Pacemaker
https://lefred.be/content/mysql-router-ha-with-pacemaker/
 MySQL Router HA with Keepalived
https://lefred.be/content/mysql-router-ha-with-keepalived/

Resources – MySQL InnoDB ReplicaSet
 MySQL InnoDB ReplicaSet Documentation
https://dev.mysql.com/doc/refman/8.0/en/mysql-innodb-replicaset-introduction.html
 MySQL Shell
https://dev.mysql.com/doc/mysql-shell/8.0/en/
 MySQL Router
https://dev.mysql.com/doc/mysql-router/8.0/en/
 Using MySQL AdminAPI
https://dev.mysql.com/doc/mysql-shell/8.0/en/mysql-innodb-replicaset.html
 MySQL Replication
https://dev.mysql.com/doc/refman/8.0/en/replication.html
 MySQL Enterprise Monitor
https://dev.mysql.com/doc/mysql-monitor/8.0/en/

Resources – MySQL NDB Cluster
 MySQL Cluster Evaluation Guide
https://www.mysql.com/why-mysql/white-papers/mysql-cluster-evaluation-guide/
 Getting Started
https://www.mysql.com/products/cluster/start.html
 Engineering blogs
https://mysqlhighavailability.com/category/mysql-cluster/
 Learn More
https://github.com/wwwted/ndb-cluster-workshop
https://mysqlmed.wordpress.com/2017/08/18/migratingimporting-ndb-to-cluster-manager-w-version-upgrade/

The world’s most popular open source database

Questions?

Thank you
Olivier Dasini
MySQL Principal Solutions Architect EMEA
Twitter : @freshdaz
Blog : http://dasini.net/blog/en

MySQL High Availability Solutions - Avoid loss of service by reducing the risk of failures

MySQL High Availability Solutions - Avoid loss of service by reducing the risk of failures

Recommended

More Related Content

What's hot (20)

Similar to MySQL High Availability Solutions - Avoid loss of service by reducing the risk of failures (20)

More from Olivier DASINI (11)

Recently uploaded (20)

MySQL High Availability Solutions - Avoid loss of service by reducing the risk of failures