I have replication set up between 1 master and 3 replicas using GTID. All 5.6.26
The problem
Replication stopped working on one of replicas and I figured out it was because somehow the replica had additional transaction_ids in the Retrieved_Gtid_Set at one point.
Found this snapshot of show slave status\G from a few months ago and saw the following:
Retrieved_Gtid_Set: 57b092f9-65d6-51e4-a041-0050562a8ed1:24880-24891,
de235c4d-7400-11e4-bca1-005018ba04a0:1-1477:2423-6880 -- <this is server uuid
I'm not sure what the :2423-6880 was doing here (or how it got there) but it's biting me right now, because the master status has exceeded the transaction_id of 2422, and the replica thinks it has already received transactions 2423-6880
See below for this same variable today
Retrieved_Gtid_Set: 57b092f9-65d6-51e4-a041-0050562a8ed1:24880-24891,
de235c4d-7400-11e4-bca1-005018ba04a0:1-6880 -- <the gap has been closed
The issue now of course is that when the master sends a GTID that is < 6880, the replica will not execute because it thinks it already has it.
Currently the master has:
+------------------+----------+--------------+------------------+---------------------------------------------+
| File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |
+------------------+----------+--------------+------------------+---------------------------------------------+
| mysql-bin.000260 | 1683 | | | de235c4d-7400-11e4-bca1-001058ba04a0:1-3161 |
+------------------+----------+--------------+------------------+---------------------------------------------+
The question
Is there a way I can update the Executed_Gtid_Set to read de235c4d-7400-11e4-bca1-005018ba04a0:3161 so that new transactions sent from the master get executed?
THANKS!
UPDATE:
It's a little ugly, but I was able to resolve the problem.
I basically ran 3720 (6880 - 3161 + 1) junk inserts to a test table on the master, which advanced the master GTID to 6881. At that point the replica began receiving the transactions from the master without any issues. That being said, transactions 2423-3161 were lost on the replica, but nothing a mysqldump and restore couldn't fix