The norecover flag in depth

Why this matters in production

norecover stops the mechanism that repairs degraded PGs, which is the most consequential thing any of these flags does.

What it stops

Recovery: restoring missing object copies using the PG log
  → the primary knows which objects a returning OSD missed
  → it sends them
  → degraded objects become clean

norecover stops that entirely.

ceph osd set norecover
ceph -s | grep -E 'degraded|recovering'

With norecover set:
  degraded PGs stay degraded
  a returning OSD does not catch up
  new degradation is not repaired
  backfill continues, since it is a different mechanism

Distinguishing the three flags

The distinction that matters:
  norebalance stops optimisation
  nobackfill and norecover stop repair

ceph osd dump | grep flags
ceph -s | grep -E 'degraded|misplaced'

Legitimate uses

# a diagnostic pause, with a scripted restore
ceph osd set norecover
sleep 300
# measure
ceph osd unset norecover

The list is short deliberately: there are very few production situations where stopping repair is the right answer.

The harm of leaving it set

norecover left set:
  every subsequent failure leaves permanent degradation
  the cluster accumulates degraded PGs
  a second failure on a degraded PG can lose data
  and the health warning is the only indication

ceph health detail | grep OSDMAP_FLAGS
ceph -s | grep degraded

- alert: CephRecoverySuppressed
  expr: ceph_health_detail{name="OSDMAP_FLAGS"} == 1
  for: 1h
  labels: { severity: page }
  annotations:
    summary: "OSD map flags set for over an hour — check for norecover"

Quiz

Production discipline

Reserve norecover for brief scripted diagnostics — it stops the repair of degraded PGs, so every failure while it is set leaves permanent degradation. When the intent is reducing background load, norebalance and the throttles achieve it without touching redundancy.

Cross-course references

• Kubernetes: suppressing self-healing is categorically different from throttling background work
• Linux: disabling an array rebuild is not the same as limiting its rate