Pausing and resuming backfill

Why this matters in production

Several flags stop data movement and they differ in what they cover. One of them stops client I/O entirely, which is why choosing deliberately matters.

The flags

ceph osd set norebalance
ceph osd set nobackfill
ceph osd set norecover
ceph osd dump | grep flags

norebalance is the one to reach for. It stops the optimisation work and leaves the durability work running, which is what makes it safe to use during business hours.

Which to use

ceph osd set norebalance
ceph -s | grep -E 'flags|misplaced'

Resuming

ceph osd unset norebalance
ceph -s
watch -n 10 'ceph -s | grep -E "misplaced|recovery"'

Confirm the movement actually resumes. A flag cleared while another remains set produces no visible change and the operator concludes the resume failed.

ceph osd dump | grep flags
# flags noout,nobackfill      ← two set, one cleared

Flags to avoid

pause stops all client I/O. It exists for maintenance requiring the cluster to be quiescent and has essentially no role in ordinary operations.

ceph osd set pause      # do not, unless you mean it
ceph osd unset pause

nodown prevents OSDs being marked down, which means genuine failures are not detected. It has a narrow use during network work where flapping is expected, and leaving it set is dangerous.

Verifying no flags are left set

ceph osd dump | grep flags
ceph health detail | grep -i OSDMAP_FLAGS

Ceph raises a health warning for set flags, which is what prevents one being forgotten indefinitely.

Quiz

Production discipline

Reach for norebalance rather than nobackfill when pausing data movement; it defers optimisation without leaving degraded PGs unrepaired. Automate setting and clearing flags on a schedule rather than relying on memory, and treat OSDMAP_FLAGS as an item with an owner.

Cross-course references

• Kubernetes: pausing a rollout while keeping failure recovery active is the same distinction
• Linux: maintenance flags that disable monitoring need the same expiry discipline