Why three monitors — and why not two, four, or seven

Why this matters in production

Monitor count is decided once, usually at build time, usually quickly, and getting it wrong produces a cluster with less resilience than everyone believes it has. The arithmetic takes two minutes.

The table

members   quorum   tolerates   notes
   1         1         0        no redundancy
   2         2         0        worse than 1: twice the failure surface
   3         2         1        the standard
   4         3         1        no gain over 3; adds even-split risk
   5         3         2        large clusters, multi-domain
   6         4         2        no gain over 5
   7         4         3        rare; usually a topology problem

Two conclusions:

Even counts never help. Four tolerates the same single failure as three. The extra member adds commit latency, another host to maintain, and the possibility of a two-two split where nothing can proceed.

Two is actively bad. Quorum is two, so either failure stops the cluster. You have doubled the number of things that can take you down.

Why not go straight to five

Five is better only when the extra tolerance is needed, and it is not free:

• Every commit needs three acknowledgements rather than two, so commit latency rises.
• Elections involve more participants.
• Another host must be maintained, patched, and monitored.

Five is warranted when: the cluster is large (roughly 500+ OSDs), when monitors must be spread across three or more failure domains with tolerance to lose a whole domain, or when maintenance is frequent enough that running at reduced tolerance is a regular state.

Changing the count safely

ceph orch apply mon --placement="3 ceph-01 ceph-02 ceph-03"
ceph quorum_status
ceph orch apply mon --placement="5 ceph-01 ceph-02 ceph-03 ceph-04 ceph-05"
ceph quorum_status

Add one at a time and confirm quorum between each. Never remove a monitor while another is already absent — check quorum_status first, every time.

The decision, in short

• Three monitors, in three distinct failure domains, for almost every cluster.
• Five when the cluster is large, spans several domains, or undergoes frequent maintenance.
• Never two, never four, never six.
• Verify placement physically, not just in ceph mon dump.

Quiz

Production discipline

Three monitors in three distinct failure domains is the right answer for almost every cluster, and the placement matters as much as the count — three in one rack is a correct number arranged uselessly. Move to five only for large clusters, genuine multi-domain requirements, or frequent maintenance, accepting the commit-latency cost knowingly. Never run two, four, or six: an even count adds a split in which nothing proceeds. And verify the physical placement against the rack layout, because ceph mon dump reports hostnames, not racks.

Cross-course references

• Ceph: Part IX (Monitor Quorum) for the operational scenarios.
• Ceph: Part IV (Failure Domains) for choosing the domains themselves.
• Ceph: Part CXXII (Small Cluster Risks) for clusters with few domains.