I have a application that performs read and write on a Postgres DB. Now, for fault tolerance, I am planning to have a failover mechanism by placing a second database in a different region. Best solution seems to be a 2 node master-master replication.
My application is already developed in a way to use only 1 master and it only switches to a secondary master, in case primary master is not available. After a fault has occurred, once the primary master comes back online, it should first replicate all transaction from the secondary master and then it should make itself available to my application, making it eventually consistent.
I believe Postgres does not have these capabilities built in. I have come across a third party open source solution - https://2ndquadrant.com/en/resources/bdr/
Now 2 questions:
a) Am I missing something here? Any pitfall I should be careful about.
b) Experience with 2ndquadrant's solution or suggestion regarding any other tool.
There are couple of doubts as I am not a DBA.
If a slave is down but master is alive, a slave will resync with the master as soon as it comes back online. Reverse is not true, if master is down and updates and inserts are performed on Replication server, then master will not sync with the database (due to loss of bidirectionality). Is my understanding OK, if yes, what needs to be done is such case: Do I need to turn my slave into master now and change status of earlier master to slave?
When the recovery of the primary master DB starts using the slave. Should I stop all write operations in Slave to make recovery faster?