CephCXXIII · Capacity and Failure PlanningCapacity and Failure Planning
The maintenance reserve is a scheduling decision
What you'll learn
- Explain why maintenance consumes no capacity by itself
- Compare the two maintenance policies and their reserves
- Assess the chance that maintenance becomes a failure
- Record the policy alongside the capacity plan
Prerequisites
None — start here.
Verified against Ceph Tentacle 20.2.x · Ceph Squid 19.2.x (supported previous) · cephadm matches the verified Ceph release · podman 4.x · csi-rbd and csi-cephfs current · RBD / CephFS / RGW current (matches Ceph release) · Linux kernel 5.15+ (5.10 minimum) · Ubuntu 24.04 LTS (Ceph host baseline) · Debian 12 (Bookworm) (Ceph host baseline) · Rocky Linux / RHEL / AlmaLinux 9.x (Ceph host baseline) · Proxmox VE 9.x (cross-course integration) · Kubernetes 1.31+ (cross-course integration) · 2026-08-18
Why this matters in production
The maintenance reserve is the only line in a capacity plan whose size is set by a policy rather than by measurement, and it costs real hardware.
Maintenance costs no capacity until it does
ceph -s
ceph orch host ls
ceph osd set-group noout ceph-04
ceph orch host maintenance enter ceph-04
With the flag set, the absent OSDs stay in the map, no PG is remapped and no byte moves. The cluster is short of redundancy, not short of space. The capacity cost appears only in the case where the host does not come back — and at that point it is exactly the failure reserve, not a separate one.
Two policies, two reserves
| Policy | Reserve | Consequence |
|---|---|---|
| Maintenance only from HEALTH_OK, one host at a time | one host | patching is blocked whenever the cluster is degraded or backfilling |
| Maintenance permitted while one host is already absent | two hosts | patch latency stays low, capacity cost is two Nths of usable |
At six hosts that is the difference between 17 and 33 per cent of usable capacity. At twenty hosts it is 5 against 10, which is why large clusters adopt the relaxed policy and small ones cannot afford to.
The chance that maintenance becomes a failure
| Trigger | What fails |
|---|---|
| First power cycle after years of uptime | drives that do not spin up |
| Firmware update to a storage controller | the controller, occasionally permanently |
| Boot device that has degraded silently | the host will not boot |
| Fans and PSUs at rest | components that do not restart |
ceph orch host maintenance exit ceph-04
ceph -s
ceph osd unset-group noout ceph-04
You cannot buy a fraction of a host, so the probability does not size the reserve. What it decides is the policy: at small N accept the strict policy and the patch latency it causes; at larger N simply hold the second host of reserve and stop scheduling around it.
Putting the policy where the plan can see it
cat >> /secure/ceph-capacity-plan.txt <<'EOF'
maintenance policy: one host at a time, only from HEALTH_OK
maintenance reserve: 0 TiB beyond the failure reserve
accepted consequence: patching blocked while backfilling; log each deferral
revisit at: 12 hosts, or after 3 deferrals in a quarter
EOF
Quiz
Knowledge check · 4 questions
Q1. How much capacity does a planned host maintenance consume while `noout` is set?
Q2. Maintenance headroom is the capacity consumed while a host is in maintenance.
Q3. Set a maintenance policy for a cluster that keeps deferring patches.
Six hosts. Firmware and kernel updates have been deferred four times this quarter because the cluster was backfilling. The team is asking to allow maintenance during degraded states.
Q4. Why does the probability of a maintenance becoming a failure not determine the size of the maintenance reserve?
Passing score: 75%. Answers are checked in this browser.
Production discipline
Record the maintenance policy next to the maintenance reserve, because one determines the other and a plan showing only the number cannot be reviewed. Log every deferred maintenance item — a pattern of deferrals is the evidence that the strict policy has stopped being affordable.
Cross-course references
- Kubernetes: a PodDisruptionBudget is the same trade written as a scheduling rule
- Linux: patch cadence and redundancy budget are the same decision in two documents