A id val
B id A_id
We are not sure of what happened but it appears that when replication occurred on a series of these two table datasets, that A failed replication, but another insert got the lost ID, and then B referenced the wrong record in A, using what should have been the correct ID.
This occurred more than once, so its not like we lost one record from A and everything is off by 1. It happened sporadically and from initial investigation is painful to unthread.
such as
original/correct: A.id = 1, A.val = 'XYZ' B.id = 1, A_id = 1
A.id = 2, A.val = 'ABC' B.id = 2, A_id = 2
bad replication: A.id = 1, A.val = 'ABC' B.id = 1, A_id = 1
A.id = 2, A.val = 'WTF' <--- some other record due to the offseting B.id = 2, A_id = 2
gets worse and worse as more replications happened.
Does it make sense that losing a record like this would cause the next record to assume its ID, but the foreign key replications would maintain their original foreign key ID and hence give us this cluster f?