CephXLI · Metadata Servers (MDS)Metadata Servers (MDS)
Active and standby MDS daemons
What you'll learn
- Distinguish the standby modes
- Configure standby daemons appropriately
- Weigh standby-replay against plain standby
- Size the MDS daemon count for a deployment
Prerequisites
None — start here.
Verified against Ceph Tentacle 20.2.x · Ceph Squid 19.2.x (supported previous) · cephadm matches the verified Ceph release · podman 4.x · csi-rbd and csi-cephfs current · RBD / CephFS / RGW current (matches Ceph release) · Linux kernel 5.15+ (5.10 minimum) · Ubuntu 24.04 LTS (Ceph host baseline) · Debian 12 (Bookworm) (Ceph host baseline) · Rocky Linux / RHEL / AlmaLinux 9.x (Ceph host baseline) · Proxmox VE 9.x (cross-course integration) · Kubernetes 1.31+ (cross-course integration) · 2026-08-18
Why this matters in production
Without a standby, an MDS failure means the filesystem is unavailable until an operator intervenes. With the right standby configuration it means a pause of seconds. The difference is a few daemons and one configuration setting.
The states
ceph fs status cephfsRANK STATE MDS ACTIVITY DNS INOS
0 active mds.a Reqs: 412 /s 1240k 1180k
0-s standby-replay mds.b Evts: 88 /s 980k 940k
STANDBY MDS
mds.c| State | Meaning |
|---|---|
active | serving a rank |
standby | idle, available to take any rank |
standby-replay | following one rank’s journal, ready to take it over quickly |
replay | replaying a journal after taking over |
resolve, reconnect, rejoin | intermediate takeover states |
Plain standby
An idle daemon that can take any rank. On takeover it must replay the entire journal from the last checkpoint, which is where the recovery time goes.
ceph fs set cephfs standby_count_wanted 1standby-replay
A daemon that continuously follows a specific rank’s journal, keeping its cache warm.
ceph fs set cephfs allow_standby_replay trueOn failover it has already replayed almost everything and takes over in a fraction of the time.
| Plain standby | standby-replay | |
|---|---|---|
| Failover time | journal replay, tens of seconds | seconds |
| Memory use while idle | minimal | comparable to the active MDS |
| Can take any rank | yes | tied to its rank |
| Metadata pool read load | none | continuous |
standby-replay is not free: it holds a warm cache, so it needs memory comparable to the active daemon, and it reads the journal continuously.
Sizing
daemons needed = max_mds + standbys| Deployment | max_mds | standbys | Total |
|---|---|---|---|
| Small, tolerant of a pause | 1 | 1 | 2 |
| Production, single rank | 1 | 1 standby-replay | 2 |
| Production, three ranks | 3 | 1–3 | 4–6 |
| Failover latency critical | n | n standby-replay | 2n |
With standby-replay and multiple ranks, one replay daemon per rank is the configuration that gives fast failover everywhere — at the cost of double the MDS memory footprint.
Quiz
Knowledge check · 4 questions
Q1. What is the main cost of running a standby-replay MDS rather than a plain standby?
Q2. A single standby-replay daemon accelerates failover for every rank in a multi-rank filesystem.
Q3. Size MDS daemons for a production deployment.
A CephFS filesystem serves an interactive application where a 60-second pause is unacceptable. Measurement shows one active rank is sufficient for the metadata load. Each MDS needs about 24 GB of cache memory.
Q4. What happens to a CephFS filesystem with max_mds 1 and no standby when the MDS process crashes?
Passing score: 75%. Answers are checked in this browser.
Production discipline
Provision at least one standby for every production CephFS filesystem; a single-daemon deployment converts a process crash into an operator-response outage. Where failover latency matters, budget the memory for standby-replay — it is the difference between tens of seconds and a few.
Cross-course references
- Kubernetes: leader election with a warm follower has the same failover-latency trade
- Linux: active-passive clustering with a pre-warmed standby follows the identical pattern