1、在master上手动添加vip:
ifcfg eno16777984:1 192.168.200.188
安装ifconfig:
yum -y install net-tools
2、执行ifconfig查看网卡信息(master节点):
3、添加master_ip_failover文件(manager节点):
vi /etc/mha/master_ip_failover
#!/usr/bin/env perl
use strict;
use warnings FATAL => 'all';
use Getopt::Long;
my (
$command, $ssh_user, $orig_master_host, $orig_master_ip,
$orig_master_port, $new_master_host, $new_master_ip, $new_master_port
);
my $vip = '192.168.200.188/24';
my $key = '1';
my $ssh_start_vip = "/usr/sbin/ifconfig eno16777984:$key $vip";
my $ssh_stop_vip = "/usr/sbin/ifconfig eno16777984:$key down";
GetOptions(
'command=s' => \$command,
'ssh_user=s' => \$ssh_user,
'orig_master_host=s' => \$orig_master_host,
'orig_master_ip=s' => \$orig_master_ip,
'orig_master_port=i' => \$orig_master_port,
'new_master_host=s' => \$new_master_host,
'new_master_ip=s' => \$new_master_ip,
'new_master_port=i' => \$new_master_port,
);
exit &main();
sub main {
print "\n\nIN SCRIPT TEST====$ssh_stop_vip==$ssh_start_vip===\n\n";
if ( $command eq "stop" || $command eq "stopssh" ) {
my $exit_code = 1;
eval {
print "Disabling the VIP on old master: $orig_master_host \n";
&stop_vip();
$exit_code = 0;
};
if ($@) {
warn "Got Error: $@\n";
exit $exit_code;
}
exit $exit_code;
}
elsif ( $command eq "start" ) {
my $exit_code = 10;
eval {
print "Enabling the VIP - $vip on the new master - $new_master_host \n";
&start_vip();
$exit_code = 0;
};
if ($@) {
warn $@;
exit $exit_code;
}
exit $exit_code;
}
elsif ( $command eq "status" ) {
print "Checking the Status of the script.. OK \n";
exit 0;
}
else {
&usage();
exit 1;
}
}
sub start_vip() {
`ssh $ssh_user\@$new_master_host \" $ssh_start_vip \"`;
}
sub stop_vip() {
return 0 unless ($ssh_user);
`ssh $ssh_user\@$orig_master_host \" $ssh_stop_vip \"`;
}
sub usage {
print
"Usage: master_ip_failover --command=start|stop|stopssh|status --orig_master_host=host --orig_master_ip=ip --orig_master_port=port --new_master_host=host --new_master_ip=ip --new_master_port=port\n";
}
注:此两处根据情况自行修改:
4、添加执行权限(manager节点)::
chmod +x /etc/mha/master_ip_failover
5、修改mha配置文件(manager节点):
vi /etc/mha/mha.cnf
[server default]
user=mhaadmin
password=mha123
port=3306
manager_workdir=/etc/mha/app1
manager_log=/etc/mha/manager.log
remote_workdir=/mysqlsoft/mha/app1
ssh_user=root
repl_user=user_slave
repl_password=123456
ping_interval=1
master_binlog_dir=/mysqlsoft/mysql/data
master_ip_failover_script=/etc/mha/master_ip_failover
[server1]
hostname=192.168.200.59
ssh_port=22
candidate_master=1
check_repl_delay=0 #忽略relay logs日志的复制延迟
[server2]
hostname=192.168.200.57
ssh_port=22
candidate_master=1
6、启动mha,查看mha运行状态(manager节点):
启动mha:
nohup masterha_manager -conf=/etc/mha/mha.cnf &> /etc/mha/manager.log &
查看mha状态:
masterha_check_status -conf=/etc/mha/mha.cnf
7、停止mysql服务(master节点):
su - mysql
mysql.server stop
停止mysql服务之后发现,master节点上的vip已没有。
8、查看manager.log(manager节点):
cat /etc/mha/manager.log
....................
....................
Thu Aug 31 10:52:01 2023 - [warning] Got error on MySQL select ping: 2006 (MySQL server has gone away)
Thu Aug 31 10:52:01 2023 - [info] Executing SSH check script: save_binary_logs --command=test --start_pos=4 --binlog_dir=/mysqlsoft/mysql/data --output_file=/mysqlsoft/mha/app1/save_binary_logs_test --manager_version=0.58 --binlog_prefix=mysql-bin
Thu Aug 31 10:52:01 2023 - [info] HealthCheck: SSH to 192.168.200.57 is reachable.
Thu Aug 31 10:52:02 2023 - [warning] Got error on MySQL connect: 2003 (Can't connect to MySQL server on '192.168.200.57' (111))
Thu Aug 31 10:52:02 2023 - [warning] Connection failed 2 time(s)..
Thu Aug 31 10:52:03 2023 - [warning] Got error on MySQL connect: 2003 (Can't connect to MySQL server on '192.168.200.57' (111))
Thu Aug 31 10:52:03 2023 - [warning] Connection failed 3 time(s)..
Thu Aug 31 10:52:04 2023 - [warning] Got error on MySQL connect: 2003 (Can't connect to MySQL server on '192.168.200.57' (111))
Thu Aug 31 10:52:04 2023 - [warning] Connection failed 4 time(s)..
Thu Aug 31 10:52:04 2023 - [warning] Master is not reachable from health checker!
Thu Aug 31 10:52:04 2023 - [warning] Master 192.168.200.57(192.168.200.57:3306) is not reachable!
Thu Aug 31 10:52:04 2023 - [warning] SSH is reachable.
Thu Aug 31 10:52:04 2023 - [info] Connecting to a master server failed. Reading configuration file /etc/masterha_default.cnf and /etc/mha/mha.cnf again, and trying to connect to all servers to check server status..
Thu Aug 31 10:52:04 2023 - [warning] Global configuration file /etc/masterha_default.cnf not found. Skipping.
Thu Aug 31 10:52:04 2023 - [info] Reading application default configuration from /etc/mha/mha.cnf..
Thu Aug 31 10:52:04 2023 - [info] Reading server configuration from /etc/mha/mha.cnf..
Thu Aug 31 10:52:05 2023 - [info] GTID failover mode = 0
Thu Aug 31 10:52:05 2023 - [info] Dead Servers:
Thu Aug 31 10:52:05 2023 - [info] 192.168.200.57(192.168.200.57:3306)
Thu Aug 31 10:52:05 2023 - [info] Alive Servers:
Thu Aug 31 10:52:05 2023 - [info] 192.168.200.59(192.168.200.59:3306)
Thu Aug 31 10:52:05 2023 - [info] Alive Slaves:
Thu Aug 31 10:52:05 2023 - [info] 192.168.200.59(192.168.200.59:3306) Version=8.0.16 (oldest major version between slaves) log-bin:enabled
Thu Aug 31 10:52:05 2023 - [info] Replicating from 192.168.200.57(192.168.200.57:3306)
Thu Aug 31 10:52:05 2023 - [info] Primary candidate for the new Master (candidate_master is set)
Thu Aug 31 10:52:05 2023 - [info] Checking slave configurations..
Thu Aug 31 10:52:05 2023 - [warning] relay_log_purge=0 is not set on slave 192.168.200.59(192.168.200.59:3306).
Thu Aug 31 10:52:05 2023 - [info] Checking replication filtering settings..
Thu Aug 31 10:52:05 2023 - [info] Replication filtering check ok.
Thu Aug 31 10:52:05 2023 - [info] Master is down!
Thu Aug 31 10:52:05 2023 - [info] Terminating monitoring script.
Thu Aug 31 10:52:05 2023 - [info] Got exit code 20 (Master dead).
Thu Aug 31 10:52:05 2023 - [info] MHA::MasterFailover version 0.58.
Thu Aug 31 10:52:05 2023 - [info] Starting master failover.
Thu Aug 31 10:52:05 2023 - [info]
Thu Aug 31 10:52:05 2023 - [info] * Phase 1: Configuration Check Phase..
Thu Aug 31 10:52:05 2023 - [info]
Thu Aug 31 10:52:06 2023 - [info] GTID failover mode = 0
Thu Aug 31 10:52:06 2023 - [info] Dead Servers:
Thu Aug 31 10:52:06 2023 - [info] 192.168.200.57(192.168.200.57:3306)
Thu Aug 31 10:52:06 2023 - [info] Checking master reachability via MySQL(double check)...
Thu Aug 31 10:52:06 2023 - [info] ok.
Thu Aug 31 10:52:06 2023 - [info] Alive Servers:
Thu Aug 31 10:52:06 2023 - [info] 192.168.200.59(192.168.200.59:3306)
Thu Aug 31 10:52:06 2023 - [info] Alive Slaves:
Thu Aug 31 10:52:06 2023 - [info] 192.168.200.59(192.168.200.59:3306) Version=8.0.16 (oldest major version between slaves) log-bin:enabled
Thu Aug 31 10:52:06 2023 - [info] Replicating from 192.168.200.57(192.168.200.57:3306)
Thu Aug 31 10:52:06 2023 - [info] Primary candidate for the new Master (candidate_master is set)
Thu Aug 31 10:52:06 2023 - [info] Starting Non-GTID based failover.
Thu Aug 31 10:52:06 2023 - [info]
Thu Aug 31 10:52:06 2023 - [info] ** Phase 1: Configuration Check Phase completed.
Thu Aug 31 10:52:06 2023 - [info]
Thu Aug 31 10:52:06 2023 - [info] * Phase 2: Dead Master Shutdown Phase..
Thu Aug 31 10:52:06 2023 - [info]
Thu Aug 31 10:52:06 2023 - [info] Forcing shutdown so that applications never connect to the current master..
Thu Aug 31 10:52:06 2023 - [info] Executing master IP deactivation script:
Thu Aug 31 10:52:06 2023 - [info] /etc/mha/master_ip_failover --orig_master_host=192.168.200.57 --orig_master_ip=192.168.200.57 --orig_master_port=3306 --command=stopssh --ssh_user=root
IN SCRIPT TEST====/usr/sbin/ifconfig eno16777984:1 down==/usr/sbin/ifconfig eno16777984:1 192.168.200.188/24===
Disabling the VIP on old master: 192.168.200.57
Thu Aug 31 10:52:06 2023 - [info] done.
Thu Aug 31 10:52:06 2023 - [warning] shutdown_script is not set. Skipping explicit shutting down of the dead master.
Thu Aug 31 10:52:06 2023 - [info] * Phase 2: Dead Master Shutdown Phase completed.
Thu Aug 31 10:52:06 2023 - [info]
Thu Aug 31 10:52:06 2023 - [info] * Phase 3: Master Recovery Phase..
Thu Aug 31 10:52:06 2023 - [info]
Thu Aug 31 10:52:06 2023 - [info] * Phase 3.1: Getting Latest Slaves Phase..
Thu Aug 31 10:52:06 2023 - [info]
Thu Aug 31 10:52:06 2023 - [info] The latest binary log file/position on all slaves is mysql-bin.000028:155
Thu Aug 31 10:52:06 2023 - [info] Latest slaves (Slaves that received relay log files to the latest):
Thu Aug 31 10:52:06 2023 - [info] 192.168.200.59(192.168.200.59:3306) Version=8.0.16 (oldest major version between slaves) log-bin:enabled
Thu Aug 31 10:52:06 2023 - [info] Replicating from 192.168.200.57(192.168.200.57:3306)
Thu Aug 31 10:52:06 2023 - [info] Primary candidate for the new Master (candidate_master is set)
Thu Aug 31 10:52:06 2023 - [info] The oldest binary log file/position on all slaves is mysql-bin.000028:155
Thu Aug 31 10:52:06 2023 - [info] Oldest slaves:
Thu Aug 31 10:52:06 2023 - [info] 192.168.200.59(192.168.200.59:3306) Version=8.0.16 (oldest major version between slaves) log-bin:enabled
Thu Aug 31 10:52:06 2023 - [info] Replicating from 192.168.200.57(192.168.200.57:3306)
Thu Aug 31 10:52:06 2023 - [info] Primary candidate for the new Master (candidate_master is set)
Thu Aug 31 10:52:06 2023 - [info]
Thu Aug 31 10:52:06 2023 - [info] * Phase 3.2: Saving Dead Master's Binlog Phase..
Thu Aug 31 10:52:06 2023 - [info]
Thu Aug 31 10:52:07 2023 - [info] Fetching dead master's binary logs..
Thu Aug 31 10:52:07 2023 - [info] Executing command on the dead master 192.168.200.57(192.168.200.57:3306): save_binary_logs --command=save --start_file=mysql-bin.000028 --start_pos=155 --binlog_dir=/mysqlsoft/mysql/data --output_file=/mysqlsoft/mha/app1/saved_master_binlog_from_192.168.200.57_3306_20230831105205.binlog --handle_raw_binlog=1 --disable_log_bin=0 --manager_version=0.58
Creating /mysqlsoft/mha/app1 if not exists.. ok.
Concat binary/relay logs from mysql-bin.000028 pos 155 to mysql-bin.000028 EOF into /mysqlsoft/mha/app1/saved_master_binlog_from_192.168.200.57_3306_20230831105205.binlog ..
Binlog Checksum enabled
Dumping binlog format description event, from position 0 to 155.. ok.
Dumping effective binlog data from /mysqlsoft/mysql/data/mysql-bin.000028 position 155 to tail(178).. ok.
Binlog Checksum enabled
Concat succeeded.
Thu Aug 31 10:52:08 2023 - [info] scp from root@192.168.200.57:/mysqlsoft/mha/app1/saved_master_binlog_from_192.168.200.57_3306_20230831105205.binlog to local:/etc/mha/app1/saved_master_binlog_from_192.168.200.57_3306_20230831105205.binlog succeeded.
Thu Aug 31 10:52:09 2023 - [info] HealthCheck: SSH to 192.168.200.59 is reachable.
Thu Aug 31 10:52:10 2023 - [info]
Thu Aug 31 10:52:10 2023 - [info] * Phase 3.3: Determining New Master Phase..
Thu Aug 31 10:52:10 2023 - [info]
Thu Aug 31 10:52:10 2023 - [info] Finding the latest slave that has all relay logs for recovering other slaves..
Thu Aug 31 10:52:10 2023 - [info] All slaves received relay logs to the same position. No need to resync each other.
Thu Aug 31 10:52:10 2023 - [info] Searching new master from slaves..
Thu Aug 31 10:52:10 2023 - [info] Candidate masters from the configuration file:
Thu Aug 31 10:52:10 2023 - [info] 192.168.200.59(192.168.200.59:3306) Version=8.0.16 (oldest major version between slaves) log-bin:enabled
Thu Aug 31 10:52:10 2023 - [info] Replicating from 192.168.200.57(192.168.200.57:3306)
Thu Aug 31 10:52:10 2023 - [info] Primary candidate for the new Master (candidate_master is set)
Thu Aug 31 10:52:10 2023 - [info] Non-candidate masters:
Thu Aug 31 10:52:10 2023 - [info] Searching from candidate_master slaves which have received the latest relay log events..
Thu Aug 31 10:52:10 2023 - [info] New master is 192.168.200.59(192.168.200.59:3306)
Thu Aug 31 10:52:10 2023 - [info] Starting master failover..
Thu Aug 31 10:52:10 2023 - [info]
From:
192.168.200.57(192.168.200.57:3306) (current master)
+--192.168.200.59(192.168.200.59:3306)
To:
192.168.200.59(192.168.200.59:3306) (new master)
Thu Aug 31 10:52:10 2023 - [info]
Thu Aug 31 10:52:10 2023 - [info] * Phase 3.4: New Master Diff Log Generation Phase..
Thu Aug 31 10:52:10 2023 - [info]
Thu Aug 31 10:52:10 2023 - [info] This server has all relay logs. No need to generate diff files from the latest slave.
Thu Aug 31 10:52:10 2023 - [info] Sending binlog..
Thu Aug 31 10:52:11 2023 - [info] scp from local:/etc/mha/app1/saved_master_binlog_from_192.168.200.57_3306_20230831105205.binlog to root@192.168.200.59:/mysqlsoft/mha/app1/saved_master_binlog_from_192.168.200.57_3306_20230831105205.binlog succeeded.
Thu Aug 31 10:52:11 2023 - [info]
Thu Aug 31 10:52:11 2023 - [info] * Phase 3.5: Master Log Apply Phase..
Thu Aug 31 10:52:11 2023 - [info]
Thu Aug 31 10:52:11 2023 - [info] *NOTICE: If any error happens from this phase, manual recovery is needed.
Thu Aug 31 10:52:11 2023 - [info] Starting recovery on 192.168.200.59(192.168.200.59:3306)..
Thu Aug 31 10:52:11 2023 - [info] Generating diffs succeeded.
Thu Aug 31 10:52:11 2023 - [info] Waiting until all relay logs are applied.
Thu Aug 31 10:52:11 2023 - [info] done.
Thu Aug 31 10:52:11 2023 - [info] Getting slave status..
Thu Aug 31 10:52:11 2023 - [info] This slave(192.168.200.59)'s Exec_Master_Log_Pos equals to Read_Master_Log_Pos(mysql-bin.000028:155). No need to recover from Exec_Master_Log_Pos.
Thu Aug 31 10:52:11 2023 - [info] Connecting to the target slave host 192.168.200.59, running recover script..
Thu Aug 31 10:52:11 2023 - [info] Executing command: apply_diff_relay_logs --command=apply --slave_user='mhaadmin' --slave_host=192.168.200.59 --slave_ip=192.168.200.59 --slave_port=3306--apply_files=/mysqlsoft/mha/app1/saved_master_binlog_from_192.168.200.57_3306_20230831105205.binlog --workdir=/mysqlsoft/mha/app1 --target_version=8.0.16 --timestamp=20230831105205 --handle_raw_binlog=1 --disable_log_bin=0 --manager_version=0.58 --slave_pass=xxx
Thu Aug 31 10:52:11 2023 - [info]
Applying differential binary/relay log files /mysqlsoft/mha/app1/saved_master_binlog_from_192.168.200.57_3306_20230831105205.binlog on 192.168.200.59:3306. This may take long time...
Applying log files succeeded.
Thu Aug 31 10:52:11 2023 - [info] All relay logs were successfully applied.
Thu Aug 31 10:52:11 2023 - [info] Getting new master's binlog name and position..
Thu Aug 31 10:52:11 2023 - [info] mysql-bin.000018:155
Thu Aug 31 10:52:11 2023 - [info] All other slaves should start replication from here. Statement should be: CHANGE MASTER TO MASTER_HOST='192.168.200.59', MASTER_PORT=3306, MASTER_LOG_FILE='mysql-bin.000018', MASTER_LOG_POS=155, MASTER_USER='user_slave', MASTER_PASSWORD='xxx';
Thu Aug 31 10:52:11 2023 - [info] Executing master IP activate script:
Thu Aug 31 10:52:11 2023 - [info] /etc/mha/master_ip_failover --command=start --ssh_user=root --orig_master_host=192.168.200.57 --orig_master_ip=192.168.200.57 --orig_master_port=3306 --new_master_host=192.168.200.59 --new_master_ip=192.168.200.59 --new_master_port=3306 --new_master_user='mhaadmin' --new_master_password=xxx
Unknown option: new_master_user
Unknown option: new_master_password
IN SCRIPT TEST====/usr/sbin/ifconfig eno16777984:1 down==/usr/sbin/ifconfig eno16777984:1 192.168.200.188/24===
Enabling the VIP - 192.168.200.188/24 on the new master - 192.168.200.59
Thu Aug 31 10:52:12 2023 - [info] OK.
Thu Aug 31 10:52:12 2023 - [info] Setting read_only=0 on 192.168.200.59(192.168.200.59:3306)..
Thu Aug 31 10:52:12 2023 - [info] ok.
Thu Aug 31 10:52:12 2023 - [info] ** Finished master recovery successfully.
Thu Aug 31 10:52:12 2023 - [info] * Phase 3: Master Recovery Phase completed.
Thu Aug 31 10:52:12 2023 - [info]
Thu Aug 31 10:52:12 2023 - [info] * Phase 4: Slaves Recovery Phase..
Thu Aug 31 10:52:12 2023 - [info]
Thu Aug 31 10:52:12 2023 - [info] * Phase 4.1: Starting Parallel Slave Diff Log Generation Phase..
Thu Aug 31 10:52:12 2023 - [info]
Thu Aug 31 10:52:12 2023 - [info] Generating relay diff files from the latest slave succeeded.
Thu Aug 31 10:52:12 2023 - [info]
Thu Aug 31 10:52:12 2023 - [info] * Phase 4.2: Starting Parallel Slave Log Apply Phase..
Thu Aug 31 10:52:12 2023 - [info]
Thu Aug 31 10:52:12 2023 - [info] All new slave servers recovered successfully.
Thu Aug 31 10:52:12 2023 - [info]
Thu Aug 31 10:52:12 2023 - [info] * Phase 5: New master cleanup phase..
Thu Aug 31 10:52:12 2023 - [info]
Thu Aug 31 10:52:12 2023 - [info] Resetting slave info on the new master..
Thu Aug 31 10:52:12 2023 - [info] 192.168.200.59: Resetting slave info succeeded.
Thu Aug 31 10:52:12 2023 - [info] Master failover to 192.168.200.59(192.168.200.59:3306) completed successfully.
Thu Aug 31 10:52:12 2023 - [info]
----- Failover Report -----
mha: MySQL Master failover 192.168.200.57(192.168.200.57:3306) to 192.168.200.59(192.168.200.59:3306) succeeded
Master 192.168.200.57(192.168.200.57:3306) is down!
Check MHA Manager logs at localhost.localdomain:/etc/mha/manager.log for details.
Started automated(non-interactive) failover.
Invalidated master IP address on 192.168.200.57(192.168.200.57:3306)
The latest slave 192.168.200.59(192.168.200.59:3306) has all relay logs for recovery.
Selected 192.168.200.59(192.168.200.59:3306) as a new master.
192.168.200.59(192.168.200.59:3306): OK: Applying all logs succeeded.
192.168.200.59(192.168.200.59:3306): OK: Activated master IP address.
Generating relay diff files from the latest slave succeeded.
192.168.200.59(192.168.200.59:3306): Resetting slave info succeeded.
Master failover to 192.168.200.59(192.168.200.59:3306) completed successfully.
[1]+ 完成 nohup masterha_manager -conf=/etc/mha/mha.cnf &>/etc/mha/manager.log