Tuning scrub deliberately

Advanced⏱ ~18 minceph

What you'll learn

Identify the settings that shape scrub behaviour
Understand how they interact
Configure scrub for a given cluster type
Verify the configuration is achieving its intent

Prerequisites

None — start here.

Verified against Ceph Tentacle 20.2.x · Ceph Squid 19.2.x (supported previous) · cephadm matches the verified Ceph release · podman 4.x · csi-rbd and csi-cephfs current · RBD / CephFS / RGW current (matches Ceph release) · Linux kernel 5.15+ (5.10 minimum) · Ubuntu 24.04 LTS (Ceph host baseline) · Debian 12 (Bookworm) (Ceph host baseline) · Rocky Linux / RHEL / AlmaLinux 9.x (Ceph host baseline) · Proxmox VE 9.x (cross-course integration) · Kubernetes 1.31+ (cross-course integration) · 2026-08-18

Why this matters in production

Scrub configuration is a set of interacting settings, and the interactions are where the mistakes live. A defensible configuration comes from choosing an intent and then setting the values that serve it.

The settings

ceph config get osd osd_max_scrubs
ceph config get osd osd_scrub_load_threshold
ceph config get osd osd_scrub_begin_hour
ceph config get osd osd_scrub_end_hour
ceph config get osd osd_scrub_sleep
ceph config get osd osd_deep_scrub_interval
ceph config get osd osd_scrub_max_interval
ceph config get osd osd_deep_scrub_stride

Setting	Shapes
`osd_max_scrubs`	concurrency per OSD
`osd_scrub_load_threshold`	whether a scrub starts under load
`osd_scrub_begin_hour` / `_end_hour`	the permitted window
`osd_scrub_sleep`	delay between scrub chunks
`osd_deep_scrub_interval`	how often bytes are verified
`osd_deep_scrub_stride`	bytes read per deep scrub chunk

Per-pool control

ceph osd pool set backups noscrub false
ceph osd pool set backups nodeep-scrub false
ceph osd pool set scratch nodeep-scrub true
ceph osd pool get rbd-vms scrub_min_interval

Per-pool settings are the sharpest tool: they let a scratch pool be scrubbed rarely while an archive pool is scrubbed on schedule, which a cluster-wide setting cannot express.

Configurations by cluster type

HDD, latency-sensitive

ceph config set osd osd_max_scrubs 1
ceph config set osd osd_scrub_begin_hour 22
ceph config set osd osd_scrub_end_hour 6
ceph config set osd osd_scrub_load_threshold 2.0
ceph config set osd osd_deep_scrub_interval 1209600
ceph config set osd osd_scrub_sleep 0.1

All-flash

ceph config set osd osd_max_scrubs 3
ceph config set osd osd_scrub_sleep 0
# no time window — flash absorbs the load

Archive, throughput-oriented

ceph config set osd osd_max_scrubs 2
ceph config set osd osd_deep_scrub_interval 604800
# scrub matters most here; cold data has no other check

Verifying the intent

# is the schedule keeping up?
ceph health detail | grep -E 'NOT_SCRUBBED|NOT_DEEP_SCRUBBED'

# how old is the oldest?
ceph pg dump pgs 2>/dev/null | awk 'NR>1 {print $23}' | sort | head -1

# is the concurrency being used?
ceph pg dump pgs 2>/dev/null | grep -c scrubbing

If no PGs are ever scrubbing during the window, the load threshold is never met and the configuration is not achieving its intent regardless of what the settings say.

Quiz

Knowledge check · 4 questions

Q1. On an HDD cluster, why is `osd_scrub_sleep` often more effective than lowering `osd_max_scrubs`?

It reduces the total amount of scrub work, because a chunk whose checksums matched on the previous pass is skipped on the next one It pauses between chunks, returning the disk head to client work at intervals, rather than leaving a stream reading continuously It disables deep scrub on spinning media while leaving the cheaper shallow scrub running as normal It applies only to flash devices, where the scheduler cannot otherwise bound scrub concurrency
Q2. A scratch pool and a compliance archive on one cluster need opposite scrub policies.

True False
Q3. Configure scrub for a mixed cluster.

A cluster hosts an all-NVMe pool for databases, an HDD pool for VM backups, and an HDD pool for a compliance archive that is read perhaps twice a year. One cluster-wide scrub configuration is in place.
Q4. How do you tell whether a scrub configuration is achieving its intent?

Passing score: 75%. Answers are checked in this browser.

Production discipline

Use per-pool scrub settings where pools have different requirements; a single cluster-wide interval serves the read-light archive and the scratch pool equally badly. On HDD reach for osd_scrub_sleep before lowering concurrency, and verify the configuration by the stamps rather than by the settings.

Cross-course references

Kubernetes: per-namespace resource policy expresses what cluster defaults cannot
Linux: per-filesystem scrub schedules serve the same differentiation