I have an app that uses local proprietary DB, Berkeley DB is the closest opensource analog for that DB.
Until today there were no requirement to run multiple nodes of this application, and right now I need to implement this support.
What I am looking for is the reference implementation of simplest replication possible for DB, i.e. my plan is to keep using the same app+local db and keep using local reads. However all writes will be synchronously replicated to all the nodes.
Here is how I see this working: I'll be running a cluster of nodes, every node has its own copy of app+local db and every node will maintain list of its peers. If certain peer becomes unresponsive, node notifies other available peers that particular peer needs to be removed and emits alarm. Recovery procedure: stop all nodes, restore network connection between nodes and then start all nodes again. And yes, manual recovery and certain period of downtime is OK for me.
So this seem to be simplest possible solution that may work for me now. Where can I find documentation/borrow code snippets/anything on this matter to implement this replication as quickly as possible?