The capacity budget: what may actually be committed

Intermediate⏱ ~17 minceph

What you'll learn

Split usable capacity into the four planning lines
Read the starting number from ceph df detail
Keep logical and raw figures apart in the plan
Record the budget so it can be checked later

Prerequisites

None — start here.

Verified against Ceph Tentacle 20.2.x · Ceph Squid 19.2.x (supported previous) · cephadm matches the verified Ceph release · podman 4.x · csi-rbd and csi-cephfs current · RBD / CephFS / RGW current (matches Ceph release) · Linux kernel 5.15+ (5.10 minimum) · Ubuntu 24.04 LTS (Ceph host baseline) · Debian 12 (Bookworm) (Ceph host baseline) · Rocky Linux / RHEL / AlmaLinux 9.x (Ceph host baseline) · Proxmox VE 9.x (cross-course integration) · Kubernetes 1.31+ (cross-course integration) · 2026-08-18

Why this matters in production

Free space and committable space are different numbers, and every capacity promise that later went wrong was made against the first one.

The four lines of the budget

Line	Meaning	Source
Live data	what the pools currently store	STORED in `ceph df`
Failure reserve	space kept free so a lost host can re-replicate	one host worth of usable
Maintenance reserve	space for a second host being absent at the same time	zero, or one more host
Growth allowance	what may be committed before the next delivery lands	whatever remains

The constraint is that live data plus growth allowance must stay at or below usable capacity minus both reserves. Everything else in a capacity plan is arithmetic on those four lines.

Reading the starting number

ceph df detail

--- RAW STORAGE ---
CLASS  SIZE     AVAIL    USED     RAW USED  %RAW USED
ssd    384 TiB  174 TiB  210 TiB   210 TiB      54.69
TOTAL  384 TiB  174 TiB  210 TiB   210 TiB      54.69

--- POOLS ---
POOL         ID  PGS  STORED   OBJECTS  USED     %USED  MAX AVAIL
rbd           2  512   68 TiB   18.31M  204 TiB  58.62     48 TiB
cephfs_data   3  256  2.0 TiB    9.44M  6.0 TiB   4.00     48 TiB

Usable capacity is STORED plus MAX AVAIL for the pools sharing a rule — 118 TiB here. That is the number the budget divides up, and it is already net of the full ratio and of the fullest OSD.

Logical against raw, which is where plans break

ceph osd df tree | head -6
ceph osd pool get rbd size

Figure	Unit
STORED, MAX AVAIL	logical — what clients wrote
USED in the POOLS section	raw — after replication or EC
Everything in `ceph osd df`	raw
RAW STORAGE totals	raw

A reserve written as 20 TB without saying which unit is wrong by the replication factor. Twenty TiB of logical reserve costs 60 TiB of raw at size 3, and raw is what appears on the purchase order.

Writing the budget where it will be checked

cat >> /secure/ceph-capacity-plan.txt <<'EOF'
2026-08-18  usable (logical)      118.0 TiB   ceph df detail, STORED + MAX AVAIL
            hosts                     6       per-host usable 19.7 TiB
            live data              70.0 TiB
            failure reserve        19.7 TiB   one host
            maintenance reserve     0.0 TiB   policy: no overlap permitted
            growth allowance       28.3 TiB
            review                2026-11-18
EOF

Quiz

Knowledge check · 4 questions

Q1. What does MAX AVAIL in `ceph df detail` already account for, and what does it not?

It accounts for the reserves recorded in the capacity plan, but not for the full ratio that stops writes It accounts for the full ratio and the fullest OSD, but for no reserve and for no growth in other pools It accounts for replication only, so the full ratio and the OSD spread still have to be subtracted by hand It accounts for everything except erasure coding, which has to be applied separately to any EC pool
Q2. MAX AVAIL is the amount of new data a cluster may safely be committed to hold.

True False
Q3. Turn a free-space figure into a committable figure.

A six-host cluster reports 70 TiB stored and 48 TiB MAX AVAIL. A team has asked whether a 40 TiB archive can be onboarded next month.
Q4. Why must a capacity plan state whether each figure is logical or raw?

Passing score: 75%. Answers are checked in this browser.

Production discipline

State usable capacity as STORED plus MAX AVAIL, then subtract the reserves before quoting anything to anyone. Write the unit next to every figure in the plan — logical or raw — because the factor between them is the replication factor and it is the most common capacity error there is.

Cross-course references

Kubernetes: allocatable node capacity is requests minus system reserved, for the same reason
Linux: filesystem free space and the space a user may consume are separated by reserved blocks