The nearfull threshold

Intermediate⏱ ~16 minceph

What you'll learn

Explain what nearfull signals
Read the nearfull health output
Distinguish OSD nearfull from pool nearfull
Respond while options remain

Prerequisites

None — start here.

Verified against Ceph Tentacle 20.2.x · Ceph Squid 19.2.x (supported previous) · cephadm matches the verified Ceph release · podman 4.x · csi-rbd and csi-cephfs current · RBD / CephFS / RGW current (matches Ceph release) · Linux kernel 5.15+ (5.10 minimum) · Ubuntu 24.04 LTS (Ceph host baseline) · Debian 12 (Bookworm) (Ceph host baseline) · Rocky Linux / RHEL / AlmaLinux 9.x (Ceph host baseline) · Proxmox VE 9.x (cross-course integration) · Kubernetes 1.31+ (cross-course integration) · 2026-08-18

Why this matters in production

Nearfull is the only capacity signal that arrives while every remedy is still available. Everything after it removes options.

What it means

ceph osd dump | grep -E 'nearfull_ratio'
ceph health detail | grep -i nearfull

[WRN] OSD_NEARFULL: 3 osds are nearfull
    osd.12 is near full
    osd.27 is near full
    osd.44 is near full

The default ratio is 0.85. At that point the OSD is still fully functional: it accepts writes, participates in backfill, and serves reads normally. Nothing is blocked.

ceph osd set-nearfull-ratio 0.85     # the current setting

OSD nearfull versus pool nearfull

ceph health detail | grep -E 'OSD_NEARFULL|POOL_NEARFULL|POOL_NEAR_FULL'

Check	Meaning
`OSD_NEARFULL`	one or more OSDs passed the ratio
`POOL_NEARFULL`	a pool is approaching its configured quota

The second is unrelated to device capacity — it is a quota set on the pool:

ceph osd pool get-quota rbd-vms
ceph osd pool set-quota rbd-vms max_bytes 50T

Confusing the two leads to adding hardware when the fix was a quota change.

Why a few OSDs and not all

ceph osd df | sort -k17 -rn | head -10
ceph osd df | awk 'NR>1 {n++; s+=$17} END {print "avg", s/n}'

Nearfull on a handful of OSDs while the average sits twenty points lower is the signature of imbalance, not of a full cluster. That distinction determines the remedy entirely:

Pattern	Remedy
A few OSDs nearfull, average much lower	balance
Most OSDs approaching together	add capacity or delete data
One OSD far above the rest	check its CRUSH weight and device size

Responding while options remain

# 1. is it imbalance?
ceph balancer status
ceph balancer eval

# 2. is there removable data?
rbd snap ls --all --pool rbd-vms
ceph df detail

# 3. is expansion already planned?
ceph orch device ls --wide

All three are available at nearfull. At backfillfull the first becomes harder; at full most become impossible.

Quiz

Knowledge check · 4 questions

Q1. What does an OSD passing the nearfull ratio change about its behaviour?

Nothing — it raises a health check and the OSD operates normally It stops accepting client writes until its utilisation falls back below the ratio It stops accepting backfill, so recovery onto it is deferred until it drains It is marked out by the monitors, so CRUSH stops mapping new PGs onto it
Q2. `POOL_NEARFULL` and `OSD_NEARFULL` both indicate device capacity pressure.

True False
Q3. Diagnose a nearfull warning.

Three of 96 OSDs report nearfull at 86%. The cluster average utilisation is 63%. A hardware purchase has been proposed in response.
Q4. Why does nearfull arrive before the thresholds that change behaviour?

Passing score: 75%. Answers are checked in this browser.

Production discipline

Compare the fullest OSD against the cluster average before treating nearfull as a capacity shortage — a wide gap means a rebalance recovers the space in hours without a purchase. Distinguish OSD_NEARFULL from POOL_NEARFULL, since the second is a quota and not a device problem.

Cross-course references

Kubernetes: a few nodes under pressure while the cluster has capacity is the same scheduling problem
Linux: an uneven LVM extent distribution produces the identical symptom