The norebalance flag in depth

Why this matters in production

norebalance is the flag that can be set during a working day without compromising anything, and understanding why makes it usable with confidence.

What it suppresses

Misplaced objects: full copy count, in locations CRUSH no longer prefers
  → produced by expansions, CRUSH changes, balancer moves, OSDs going out
  → moving them is optimisation, not repair

norebalance stops that movement and nothing else.

ceph osd set norebalance
ceph -s | grep -E 'misplaced|degraded|recovering'

With norebalance set:
  misplaced objects stay misplaced
  degraded objects are still recovered
  degraded PGs still backfill if their log gap requires it
  no redundancy is compromised at any point

Why it is safe

A misplaced object has size copies.
Leaving it misplaced does not reduce that count.
A failure while it is misplaced is handled exactly as any other failure.

The only cost is that the cluster’s distribution stays suboptimal for longer — capacity is less evenly used and MAX AVAIL stays lower than it could be.

ceph df | grep MAX
ceph osd df | sort -k17 -rn | head -3

Routine pacing

# pause an expansion during business hours
0 8  * * * ceph osd set norebalance
0 20 * * * ceph osd unset norebalance

# or pause it entirely while investigating something
ceph osd set norebalance
# investigate
ceph osd unset norebalance

Legitimate routine uses:
  pausing an expansion during peak hours
  pausing balancer movement during an incident investigation
  freezing the distribution while diagnosing a latency problem
  reducing background load during a load test

The limits

ceph -s | grep -E 'misplaced|degraded'
ceph osd df | awk 'NR>1 {if($17+0>m)m=$17+0; if(mn==""||$17+0<mn)mn=$17+0}
  END {printf "spread %.1f\n", m-mn}'

A growing spread while norebalance is set is the signal it has been left too long.

Quiz

Production discipline

Use norebalance freely for routine pacing — it suppresses only the movement of objects that already have their full copy count, so nothing is compromised. Watch the utilisation spread; a growing spread is the signal it has been left set too long.

Cross-course references

• Kubernetes: pausing optimisation-only reconciliation is likewise low-risk
• Linux: deferring defragmentation is safe where deferring a rebuild is not