SlideShare a Scribd company logo
Reliable MySQL Using Replication Issac Goldstand Mirimar Networks www.mirimar.net [email_address]
What is replication? Allows 2 or more databases to maintain state between them MySQL 3.23 and up supports asynchronous replication One server acts as a master for one or more slaves Slaves pull data to be replicated from server’s binary logs and execute the relevant statements locally
Replication basics… One master supports multiple slaves Each node on the network is assigned a unique identifying number Each slave attempts to connect to the master and fetches data to be replicated Master Slave Slave Slave Slaves
Replication basics… Clients perform data modification on master server INSERT, SELECT, DELETE, LOAD DATA EXECUTE in MySQL 5.0 and above DATA DATA INSERT INTO … Master Slave Client
Replication basics… Immediately following execution of command on master, the command is written to the local binary log Additionally, the master records its unique ID (to prevent endless loops in circular replication scenarios) and the timestamp for use with statements which use NOW(), etc. DATA DATA INSERT INTO … Binary Log Master Slave Client
Replication basics… If the slave is online, the command is transmitted to the slave in parallel (well, immediately following) to being written in the local binary log Otherwise, when the slave next connects it will receive a list of all pending statements from the master server’s binary log The slave’s replication IO thread stores the command in the local relay log DATA DATA INSERT INTO … INSERT INTO … Relay Log Replication Thread Master Slave Client
Replication basics… Once the data is received in the slave’s relay log, the slave SQL thread executes the command locally, bringing the slave up-to-date with the master In MySQL 3.23, the IO and SQL threads were just one thread.  In later versions this was changed to boost performance DATA DATA INSERT INTO … INSERT INTO … Relay Log Replication Thread DATA Master Slave Client
Replication Strategies Load balancing – single write, distributed read Master Slave Slave Slave Client (SELECT) Client (SELECT) Client (SELECT) Client (SELECT) Client (SELECT) Client (SELECT) Client (INSERT)
Replication Strategies Load balancing – single write, distributed read Load balancing – circular read/write MySQL Server MySQL Server MySQL Server Client INSERT INTO… INSERT INTO … Client
Replication Strategies Load balancing – single write, distributed read Load balancing – circular read/write High availability (hot failover) DATA DATA DATA DATA DATA Client Master Slave
Replication Strategies Load balancing – single write, distributed read Load balancing – circular read/write High availability (hot failover) Snapshot backups Master Slave Backup with LOCK TABLES (or single transaction) Client INSERT / SELECT Client INSERT / SELECT
Simple Replication Setup Modify my.cnf to include a unique  server-id  for each node On master server, ensure that  log-bin  (binary logging) is enabled in my.cnf On slave, configure login credentials on master, either via my.cnf or  CHANGE MASTER TO  statement Copy initial data snapshot from master to slave Configure initial binary log position on slave Start replication with  SLAVE START  command
Configure master my.cnf ------------- [mysqld] server-id = 1 log-bin
Configure slave my.cnf ------------- [mysqld] server-id = 2 master-user = someuser master-password = secret master-host = ip.of.master
Initial dataset Binary log provides a record of all modifications to master database starting from a fixed point: when binary logging was activated If all binary logs exist on master from initial install of MySQL, the slave(s) can use these to bring themselves up-to-date Otherwise, a snapshot of the master must be taken, using  mysqldump –master-data , to provide an initial dataset for the slave(s) If only MyISAM tables are used, the  LOAD DATA FROM MASTER  statement may be used on the slave(s)
Configure log position MASTER mysql> SHOW MASTER STATUS; +---------------------------+----------+--------------+------------------+ | File  | Position | Binlog_Do_DB | Binlog_Ignore_DB | +---------------------------+----------+--------------+------------------+ |  vmware-mirimar-bin.000002  |  79  |  |  | +---------------------------+----------+--------------+------------------+ SLAVE mysql> CHANGE MASTER TO MASTER_LOG_FILE=‘ vmware-mirimar-bin.000002 ’, MASTER_LOG_POS= 79 ; SLAVE mysql> START SLAVE;
Using CHANGE MASTER TO MASTER_HOST MASTER_USER MASTER_PASSWORD MASTER_LOG_FILE MASTER_LOG_POS
Advanced CHANGE MASTER TO MASTER_PORT RELAY_LOG_FILE RELAY_LOG_POS MASTER_SSL SSL: CA/CAPATH/CERT/KEY/CIPHER
Confirming it works SLAVE mysql> SHOW SLAVE STATUS\G *************************** 1. row *************************** Slave_IO_State: Waiting for master to send event Master_Host:  vmware-mirimar Master_User:  someuser Master_Port: 3306 Connect_Retry: 60 Master_Log_File:  vmware-mirimar-bin.000002 Read_Master_Log_Pos:  79 Relay_Log_File: vmware1-mirimar-relay-bin.000002 Relay_Log_Pos: 250 Relay_Master_Log_File: vmware-mirimar-bin.000002 Slave_IO_Running:  Yes Slave_SQL_Running:  Yes Last_Errno: 0 Last_Error: Skip_Counter: 0 Exec_Master_Log_Pos: 79 Relay_Log_Space: 250 Seconds_Behind_Master:  0
Permissions Slaves need  REPLICATION SLAVE  permission on master for basic usage If  LOAD TABLE FROM MASTER  or  LOAD DATA FROM MASTER  statements are used, slave will also need  SUPER  and  RELOAD  privileges
Internal Threads Since MySQL 4.0, replication slaves run two threads IO thread continuously receives updates from master and writes to local relay log SQL thread continuously executes statements in relay log
IO thread isolation Isolating IO thread means that slave won’t have to wait for long-executing statements to finish executing before retrieving data from master Also, slave will continue reading data from master if a statement creates a data conflict
SQL thread isolation SQL thread isolation allows for replication in an environment without a continuous link between slave and masters If master fails (or slave simply has no access), the IO thread will try to reconnect endlessly (waiting 60 seconds between attempts) SQL thread will continue processing relay logs even while IO thread is unable to connect to master
Master Thread Additionally, the master server runs the Binlog Dump thread This thread is simply dedicated to scanning the binary logs on the master and sending updates to the connected slave If this thread isn’t running, it means that replication isn’t running – more accurately, that no slaves are currently connected
Status files 2 status files for replication’s use Their use is to record the state of replication between server shutdown and startup master.info records information about the slave’s master server relay-log.info records information about the local relay logs
Information in master.info Master log file Read master log pos Master Host Master User Password (will not be shown in SHOW SLAVE STATUS) Master Port Connect Retry In MySQL 4.1+, SSL options are stored if SSL is used
Information in relay-log.info Relay log file Relay log pos Relay master-log pos Exec master-log pos
Backup master Master backups can be accomplished with mysqldump Care must be taken to ensure the following 2 special considerations: Consistent snapshot of master date (via lock tables for MyISAM or single transaction for InnoDB) Recording of binary log information, for use on slaves (master-data)
Backup master files If a file-system level backup is required, care should be taken to manually record binary log name and position via SHOW MASTER STATUS statement. To ensure consistency between backup and binary log position, the tables should be locked via FLUSH TABLES WITH READ LOCK immediately before backup (and SHOW MASTER STATUS) LEAVE THE CLIENT CONNECTED!!! After backup finishes, execute UNLOCK TABLES to release the read lock
Backup slave Same idea as master file system backup Instead of recording position, it’s enough to backup the master.info and relay-log.info files Instead of acquiring global read lock, it’s enough to STOP SLAVE before backup and START SLAVE once backup finishes
Live demo Time permitting, we’ll show a short demonstration of a simple unidirectional replication setup
For more information MySQL documentation 5.0 documentation  http://mirror.mirimar.net/mysql/doc/refman/5.0/en/replication.html 4.1 documentation  http://mirror.mirimar.net/mysql/doc/refman/4.1/en/replication.html
Thank You! For more information: Issac Goldstand margol@mirimar.net http://www.beamartyr.net/ http://www.mirimar.net/

More Related Content

PPT
Project Report Format
PPT
How to successfully give a seminar presentation
PPTX
How to Cancel your Subscription
PDF
Download disabled slideshare ppts, Download from slideshare without login
PPTX
How to present a seminar
PPTX
Slideshare.Com Powerpoint
PPT
How to review a journal paper and prepare oral presentation
PPT
Introduction to medical transcription
Project Report Format
How to successfully give a seminar presentation
How to Cancel your Subscription
Download disabled slideshare ppts, Download from slideshare without login
How to present a seminar
Slideshare.Com Powerpoint
How to review a journal paper and prepare oral presentation
Introduction to medical transcription

What's hot (20)

PPT
Computer hardware component. ppt
PPTX
power point presentation
PDF
M.tech seminar
PPTX
Smart art examples
PPTX
Logging in and Logging out of Linux - R.D.Sivakumar
PPT
Computer basics
PPTX
PDF
Video editing presentation
PPTX
Types of computer network
PPTX
Memory / Storage Devices
PPTX
Multimedia presentation
PPTX
Microsoft office
PPTX
Introduction to SQL
PPTX
Hardware & Software
PPTX
RAM(Random Access Memory)
PPTX
PPT hard disk Drive
PPTX
Ram & rom
PDF
Library management
PPTX
Computer animation
PPT
Computer Fundamentals
Computer hardware component. ppt
power point presentation
M.tech seminar
Smart art examples
Logging in and Logging out of Linux - R.D.Sivakumar
Computer basics
Video editing presentation
Types of computer network
Memory / Storage Devices
Multimedia presentation
Microsoft office
Introduction to SQL
Hardware & Software
RAM(Random Access Memory)
PPT hard disk Drive
Ram & rom
Library management
Computer animation
Computer Fundamentals
Ad

Viewers also liked (20)

PDF
What is Artificial Intelligence | Artificial Intelligence Tutorial For Beginn...
PPTX
Lengua y literatura tercero
PPTX
Endapan Placer
PDF
Presentation deduplication backup software and system
PDF
Market Research Report : Power backup market in india 2013
PPT
Windows 10 deployment training for system builders
PPTX
Lesson 5 ms office word 2007
PPTX
Windows 10 VDI Migration in real life - E2EVC Rome
PPT
Windows 8 vs windows 7 ppt
PPTX
Comparison of Windows 7 & Windows 8
PPT
Presentation on backup and recoveryyyyyyyyyyyyy
PPTX
Thalassemia dr.k.v.giridhar
PDF
Chemistry Investigatory Project Class 12 - Green Chemistry - Bio Diesel And B...
PPTX
Thalassemia
PPTX
thalassemia
PPTX
Introduction to microsoft word 2007
PPTX
01 microsoft office word 2007 (introduction and parts)
PPTX
Windows 10
PPTX
Thalassemia
PDF
Chemistry Investigatory Project (Class 12 ,CBSE)
What is Artificial Intelligence | Artificial Intelligence Tutorial For Beginn...
Lengua y literatura tercero
Endapan Placer
Presentation deduplication backup software and system
Market Research Report : Power backup market in india 2013
Windows 10 deployment training for system builders
Lesson 5 ms office word 2007
Windows 10 VDI Migration in real life - E2EVC Rome
Windows 8 vs windows 7 ppt
Comparison of Windows 7 & Windows 8
Presentation on backup and recoveryyyyyyyyyyyyy
Thalassemia dr.k.v.giridhar
Chemistry Investigatory Project Class 12 - Green Chemistry - Bio Diesel And B...
Thalassemia
thalassemia
Introduction to microsoft word 2007
01 microsoft office word 2007 (introduction and parts)
Windows 10
Thalassemia
Chemistry Investigatory Project (Class 12 ,CBSE)
Ad

Similar to Download presentation (20)

PPT
Download presentation
PPT
Download presentation531
PPT
Mysql replication @ gnugroup
PDF
2012 scale replication
PDF
2012 replication
PDF
MySQL Replication Basics -Ohio Linux Fest 2016
PDF
MySQL Replication Troubleshooting for Oracle DBAs
PPTX
mysql replication
PPT
MySQL 5.1 Replication
PDF
Replication tutorial presentation
PDF
MySQL Replication Update -- Zendcon 2016
PDF
MySQL User Camp: Multi-threaded Slaves
PPTX
MySQL Replication Overview -- PHPTek 2016
PDF
Mysql Replication Excerpt 5.1 En
PPT
Basic Knowledge on MySql Replication
DOCX
Mater,slave on mysql
PPTX
MySQL Replication Evolution -- Confoo Montreal 2017
PPT
MySQL Replication Basics
PPTX
ConFoo MySQL Replication Evolution : From Simple to Group Replication
PDF
MySQL database replication
Download presentation
Download presentation531
Mysql replication @ gnugroup
2012 scale replication
2012 replication
MySQL Replication Basics -Ohio Linux Fest 2016
MySQL Replication Troubleshooting for Oracle DBAs
mysql replication
MySQL 5.1 Replication
Replication tutorial presentation
MySQL Replication Update -- Zendcon 2016
MySQL User Camp: Multi-threaded Slaves
MySQL Replication Overview -- PHPTek 2016
Mysql Replication Excerpt 5.1 En
Basic Knowledge on MySql Replication
Mater,slave on mysql
MySQL Replication Evolution -- Confoo Montreal 2017
MySQL Replication Basics
ConFoo MySQL Replication Evolution : From Simple to Group Replication
MySQL database replication

More from webhostingguy (20)

PPT
File Upload
PDF
Running and Developing Tests with the Apache::Test Framework
PDF
MySQL and memcached Guide
PPT
Novell® iChain® 2.3
PDF
Load-balancing web servers Load-balancing web servers
PDF
SQL Server 2008 Consolidation
PDF
What is mod_perl?
PDF
What is mod_perl?
PDF
Master Service Agreement
PPT
PPT
PHP and MySQL PHP Written as a set of CGI binaries in C in ...
PDF
Dell Reference Architecture Guide Deploying Microsoft® SQL ...
PPT
Managing Diverse IT Infrastructure
PPT
Web design for business.ppt
PPS
IT Power Management Strategy
PPS
Excel and SQL Quick Tricks for Merchandisers
PPT
OLUG_xen.ppt
PPT
Parallels Hosting Products
PPT
Microsoft PowerPoint presentation 2.175 Mb
PDF
Reseller's Guide
File Upload
Running and Developing Tests with the Apache::Test Framework
MySQL and memcached Guide
Novell® iChain® 2.3
Load-balancing web servers Load-balancing web servers
SQL Server 2008 Consolidation
What is mod_perl?
What is mod_perl?
Master Service Agreement
PHP and MySQL PHP Written as a set of CGI binaries in C in ...
Dell Reference Architecture Guide Deploying Microsoft® SQL ...
Managing Diverse IT Infrastructure
Web design for business.ppt
IT Power Management Strategy
Excel and SQL Quick Tricks for Merchandisers
OLUG_xen.ppt
Parallels Hosting Products
Microsoft PowerPoint presentation 2.175 Mb
Reseller's Guide

Download presentation

  • 1. Reliable MySQL Using Replication Issac Goldstand Mirimar Networks www.mirimar.net [email_address]
  • 2. What is replication? Allows 2 or more databases to maintain state between them MySQL 3.23 and up supports asynchronous replication One server acts as a master for one or more slaves Slaves pull data to be replicated from server’s binary logs and execute the relevant statements locally
  • 3. Replication basics… One master supports multiple slaves Each node on the network is assigned a unique identifying number Each slave attempts to connect to the master and fetches data to be replicated Master Slave Slave Slave Slaves
  • 4. Replication basics… Clients perform data modification on master server INSERT, SELECT, DELETE, LOAD DATA EXECUTE in MySQL 5.0 and above DATA DATA INSERT INTO … Master Slave Client
  • 5. Replication basics… Immediately following execution of command on master, the command is written to the local binary log Additionally, the master records its unique ID (to prevent endless loops in circular replication scenarios) and the timestamp for use with statements which use NOW(), etc. DATA DATA INSERT INTO … Binary Log Master Slave Client
  • 6. Replication basics… If the slave is online, the command is transmitted to the slave in parallel (well, immediately following) to being written in the local binary log Otherwise, when the slave next connects it will receive a list of all pending statements from the master server’s binary log The slave’s replication IO thread stores the command in the local relay log DATA DATA INSERT INTO … INSERT INTO … Relay Log Replication Thread Master Slave Client
  • 7. Replication basics… Once the data is received in the slave’s relay log, the slave SQL thread executes the command locally, bringing the slave up-to-date with the master In MySQL 3.23, the IO and SQL threads were just one thread. In later versions this was changed to boost performance DATA DATA INSERT INTO … INSERT INTO … Relay Log Replication Thread DATA Master Slave Client
  • 8. Replication Strategies Load balancing – single write, distributed read Master Slave Slave Slave Client (SELECT) Client (SELECT) Client (SELECT) Client (SELECT) Client (SELECT) Client (SELECT) Client (INSERT)
  • 9. Replication Strategies Load balancing – single write, distributed read Load balancing – circular read/write MySQL Server MySQL Server MySQL Server Client INSERT INTO… INSERT INTO … Client
  • 10. Replication Strategies Load balancing – single write, distributed read Load balancing – circular read/write High availability (hot failover) DATA DATA DATA DATA DATA Client Master Slave
  • 11. Replication Strategies Load balancing – single write, distributed read Load balancing – circular read/write High availability (hot failover) Snapshot backups Master Slave Backup with LOCK TABLES (or single transaction) Client INSERT / SELECT Client INSERT / SELECT
  • 12. Simple Replication Setup Modify my.cnf to include a unique server-id for each node On master server, ensure that log-bin (binary logging) is enabled in my.cnf On slave, configure login credentials on master, either via my.cnf or CHANGE MASTER TO statement Copy initial data snapshot from master to slave Configure initial binary log position on slave Start replication with SLAVE START command
  • 13. Configure master my.cnf ------------- [mysqld] server-id = 1 log-bin
  • 14. Configure slave my.cnf ------------- [mysqld] server-id = 2 master-user = someuser master-password = secret master-host = ip.of.master
  • 15. Initial dataset Binary log provides a record of all modifications to master database starting from a fixed point: when binary logging was activated If all binary logs exist on master from initial install of MySQL, the slave(s) can use these to bring themselves up-to-date Otherwise, a snapshot of the master must be taken, using mysqldump –master-data , to provide an initial dataset for the slave(s) If only MyISAM tables are used, the LOAD DATA FROM MASTER statement may be used on the slave(s)
  • 16. Configure log position MASTER mysql> SHOW MASTER STATUS; +---------------------------+----------+--------------+------------------+ | File | Position | Binlog_Do_DB | Binlog_Ignore_DB | +---------------------------+----------+--------------+------------------+ | vmware-mirimar-bin.000002 | 79 | | | +---------------------------+----------+--------------+------------------+ SLAVE mysql> CHANGE MASTER TO MASTER_LOG_FILE=‘ vmware-mirimar-bin.000002 ’, MASTER_LOG_POS= 79 ; SLAVE mysql> START SLAVE;
  • 17. Using CHANGE MASTER TO MASTER_HOST MASTER_USER MASTER_PASSWORD MASTER_LOG_FILE MASTER_LOG_POS
  • 18. Advanced CHANGE MASTER TO MASTER_PORT RELAY_LOG_FILE RELAY_LOG_POS MASTER_SSL SSL: CA/CAPATH/CERT/KEY/CIPHER
  • 19. Confirming it works SLAVE mysql> SHOW SLAVE STATUS\G *************************** 1. row *************************** Slave_IO_State: Waiting for master to send event Master_Host: vmware-mirimar Master_User: someuser Master_Port: 3306 Connect_Retry: 60 Master_Log_File: vmware-mirimar-bin.000002 Read_Master_Log_Pos: 79 Relay_Log_File: vmware1-mirimar-relay-bin.000002 Relay_Log_Pos: 250 Relay_Master_Log_File: vmware-mirimar-bin.000002 Slave_IO_Running: Yes Slave_SQL_Running: Yes Last_Errno: 0 Last_Error: Skip_Counter: 0 Exec_Master_Log_Pos: 79 Relay_Log_Space: 250 Seconds_Behind_Master: 0
  • 20. Permissions Slaves need REPLICATION SLAVE permission on master for basic usage If LOAD TABLE FROM MASTER or LOAD DATA FROM MASTER statements are used, slave will also need SUPER and RELOAD privileges
  • 21. Internal Threads Since MySQL 4.0, replication slaves run two threads IO thread continuously receives updates from master and writes to local relay log SQL thread continuously executes statements in relay log
  • 22. IO thread isolation Isolating IO thread means that slave won’t have to wait for long-executing statements to finish executing before retrieving data from master Also, slave will continue reading data from master if a statement creates a data conflict
  • 23. SQL thread isolation SQL thread isolation allows for replication in an environment without a continuous link between slave and masters If master fails (or slave simply has no access), the IO thread will try to reconnect endlessly (waiting 60 seconds between attempts) SQL thread will continue processing relay logs even while IO thread is unable to connect to master
  • 24. Master Thread Additionally, the master server runs the Binlog Dump thread This thread is simply dedicated to scanning the binary logs on the master and sending updates to the connected slave If this thread isn’t running, it means that replication isn’t running – more accurately, that no slaves are currently connected
  • 25. Status files 2 status files for replication’s use Their use is to record the state of replication between server shutdown and startup master.info records information about the slave’s master server relay-log.info records information about the local relay logs
  • 26. Information in master.info Master log file Read master log pos Master Host Master User Password (will not be shown in SHOW SLAVE STATUS) Master Port Connect Retry In MySQL 4.1+, SSL options are stored if SSL is used
  • 27. Information in relay-log.info Relay log file Relay log pos Relay master-log pos Exec master-log pos
  • 28. Backup master Master backups can be accomplished with mysqldump Care must be taken to ensure the following 2 special considerations: Consistent snapshot of master date (via lock tables for MyISAM or single transaction for InnoDB) Recording of binary log information, for use on slaves (master-data)
  • 29. Backup master files If a file-system level backup is required, care should be taken to manually record binary log name and position via SHOW MASTER STATUS statement. To ensure consistency between backup and binary log position, the tables should be locked via FLUSH TABLES WITH READ LOCK immediately before backup (and SHOW MASTER STATUS) LEAVE THE CLIENT CONNECTED!!! After backup finishes, execute UNLOCK TABLES to release the read lock
  • 30. Backup slave Same idea as master file system backup Instead of recording position, it’s enough to backup the master.info and relay-log.info files Instead of acquiring global read lock, it’s enough to STOP SLAVE before backup and START SLAVE once backup finishes
  • 31. Live demo Time permitting, we’ll show a short demonstration of a simple unidirectional replication setup
  • 32. For more information MySQL documentation 5.0 documentation http://mirror.mirimar.net/mysql/doc/refman/5.0/en/replication.html 4.1 documentation http://mirror.mirimar.net/mysql/doc/refman/4.1/en/replication.html
  • 33. Thank You! For more information: Issac Goldstand [email protected] http://www.beamartyr.net/ http://www.mirimar.net/