Per-pool recovery priority

Why this matters in production

A cluster hosting VM disks, a backup archive, and a scratch pool recovers all three at once by default. The VM pool recovering last is a business outcome that nobody chose.

Setting the priority

ceph osd pool set rbd-vms recovery_priority 10
ceph osd pool set backups recovery_priority -5
ceph osd pool get rbd-vms recovery_priority

Higher is more urgent. The value is added to the PG’s computed recovery priority, so it biases the ordering rather than establishing a strict ordering.

ceph osd pool ls detail | grep -E 'pool|recovery_priority'

What it influences

The degradation level still dominates: a PG below min_size in a low-priority pool is recovered ahead of a merely degraded PG in a high-priority one. The setting breaks ties, it does not invert the safety ordering.

Choosing values that reflect impact

Setting these once, in advance, means the ordering during an incident reflects a decision made calmly rather than one made at 03:00.

Verifying

# a recovering PG id from the dump above:
PGID=3.1f

ceph pg dump pgs | awk '{print $1, $10}' | head -20
ceph pg "$PGID" query | grep -A5 recovery_state

During a recovery, checking which pools’ PGs are progressing first confirms whether the priorities are having the intended effect.

Quiz

Production discipline

Set recovery_priority on every pool at creation time, with the reasoning documented; the values encode which workloads matter and the ordering during an incident then needs no decision. Verify during the next recovery that the ordering matches the intent.

Cross-course references

• Kubernetes: priority classes express the same ordering intent for scheduling
• Linux: I/O priorities via ionice serve the same purpose for competing jobs