Working the numbers to an order

Advanced⏱ ~18 minceph

What you'll learn

Assemble the worksheet from cluster figures
Work an example through to an order date and quantity
Compute the placement group budget as a second constraint
Present both constraints together

Prerequisites

None — start here.

Verified against Ceph Tentacle 20.2.x · Ceph Squid 19.2.x (supported previous) · cephadm matches the verified Ceph release · podman 4.x · csi-rbd and csi-cephfs current · RBD / CephFS / RGW current (matches Ceph release) · Linux kernel 5.15+ (5.10 minimum) · Ubuntu 24.04 LTS (Ceph host baseline) · Debian 12 (Bookworm) (Ceph host baseline) · Rocky Linux / RHEL / AlmaLinux 9.x (Ceph host baseline) · Proxmox VE 9.x (cross-course integration) · Kubernetes 1.31+ (cross-course integration) · 2026-08-18

Why this matters in production

A capacity plan that does not end in a date and a quantity is a description. These are the lines that turn it into an order.

The worksheet

Line	Where it comes from
Usable capacity U	STORED plus MAX AVAIL in `ceph df detail`
Hosts N, per-host usable u	u equals U divided by N
Live data	STORED across the pools
Failure reserve	u, or twice u if the policy allows overlap
Allocatable	u times N minus one
Growth allowance	allocatable minus live data
Daily growth	measured drift plus committed step changes
Days to reserve crossing	allowance divided by daily growth
Lead-time chain	decision through to completed backfill
Order date	today plus days to crossing minus the chain

Working an example

U = 118 TiB over 6 hosts          u = 19.7 TiB per host
live = 70 TiB                     growth = 180 GiB/day
allocatable = 19.7 x 5          = 98.3 TiB
allowance   = 98.3 - 70         = 28.3 TiB
days to reserve crossing        = 28.3 x 1024 / 180 = 161 days
lead-time chain                 = 75 days
order by                        = day 86

Each host added contributes exactly u to the allowance, because the reserve also stays at one host worth.

Hosts added	Allowance	Days of runway
0	28.3 TiB	161
1	48.0 TiB	273
2	67.7 TiB	385
3	87.3 TiB	497

Two hosts buys roughly a year. That is the quantity.

The second budget, counted in placement groups

ceph config get mon mon_target_pg_per_osd
ceph config get mon mon_max_pg_per_osd
ceph config get osd osd_max_pg_per_osd_hard_ratio
ceph osd pool autoscale-status

ceph osd dump -f json | python3 -c '
import sys,json
d = json.load(sys.stdin)
osds = len([o for o in d["osds"] if o["in"]])
tot  = sum(p["pg_num"] * p["size"] for p in d["pools"])
print("pools %d  pg replicas %d  osds %d  per osd %.1f" %
      (len(d["pools"]), tot, osds, tot/osds))'

Cluster	Budget at 100 per OSD	Seven RGW pools at 32 PGs, size 3
3 hosts, 9 OSDs	900 PG replicas	672 — three quarters of the budget
6 hosts, 48 OSDs	4800 PG replicas	672 — about a seventh

What the answer looks like

Order 2 hosts by 2026-11-12. Runway after backfill: 385 days.
Constraint checked: PG budget 3075 replicas over 48 OSDs, 64 per OSD.
Adding RGW would add 672 replicas, taking it to 78 per OSD. Fits.

Why pool count rather than data volume sets the PG budget on small clusters

Every pool needs a minimum pg_num to spread across the OSDs at all, and the autoscaler will not take a pool below that floor however little data it holds. RGW is not one pool but a set — the realm root, control, metadata, log, bucket index, non-EC, and data pools — so enabling object storage adds seven PG floors that are paid whether or not a single object is written. On 48 OSDs those floors are a rounding error against the budget. On nine they are most of it, and the cluster refuses the pool for reasons that have nothing to do with how much data was going to be stored. That is why the PG budget belongs in the capacity plan beside the bytes, and why plenty of space is not an answer to whether a new service fits.

Quiz

Knowledge check · 4 questions

Q1. On a six-host cluster with 19.7 TiB usable per host and 70 TiB live, what is the growth allowance?

118 TiB, which is the full usable capacity of the cluster before any reserve is taken out 48 TiB, the MAX AVAIL figure that ceph df already reports for the pool 19.7 TiB, one host worth, which is exactly what the failure reserve holds 28.3 TiB — allocatable is u times five, and live data is subtracted from that
Q2. Free capacity and the placement group budget are independent constraints, and either can block a new pool.

True False
Q3. Produce an order date and a quantity from cluster figures.

Six hosts, 118 TiB usable, 70 TiB live, growing 180 GiB per day. The lead-time chain measures 75 days. The team wants at least a year of runway after the new capacity is in service.
Q4. Why can a small cluster refuse a new pool while holding plenty of free space?

Passing score: 75%. Answers are checked in this browser.

Production discipline

End every capacity review with a date and a quantity, and show the identity that produced them — allocatable is one host worth times the host count minus one. Check the placement group budget in the same review, because the refusal it produces arrives during a deployment rather than during planning.

Cross-course references

Kubernetes: pods per node is a count limit that binds long before CPU or memory does
Linux: inode exhaustion refuses writes on a filesystem with free blocks for the same reason