I'm baffled by the fact that this is addressed in the official docs (or at my inability to track it down). From the 3.4 docs:
Only add an arbiter to sets with even numbers of voting members. If you add an arbiter to a set with an odd number of voting members, the set may suffer from tied elections.
So we have two possible scenarios:
- A replica set with an odd number of data nodes, which requires no arbiter (and in fact, adding an arbiter can be detrimental per above)
- A replica set with an even number of data nodes, and an arbiter
This is great so long as things are running smoothly. But what if a data node goes down in each of these scenarios?
- We now have an even number of nodes with no arbiter. As I understand it this means you cannot have a successful election.
- We now have an odd number of nodes, plus an arbiter. Above docs state that this can lead to tied elections.
This is so painfully obvious, why can't I find any discussion or documentation on it? But what I really want to know is, how do I correctly handle these scenarios?