Pool autoscaling — letting the manager size pg_num

Intermediate⏱ ~16 minceph

What you'll learn

Explain how the autoscaler computes a recommendation
Adopt the autoscaler on an established cluster safely
Use target_size_ratio to guide it
Decide when to override a recommendation

Prerequisites

None — start here.

Verified against Ceph Tentacle 20.2.x · Ceph Squid 19.2.x (supported previous) · cephadm matches the verified Ceph release · podman 4.x · csi-rbd and csi-cephfs current · RBD / CephFS / RGW current (matches Ceph release) · Linux kernel 5.15+ (5.10 minimum) · Ubuntu 24.04 LTS (Ceph host baseline) · Debian 12 (Bookworm) (Ceph host baseline) · Rocky Linux / RHEL / AlmaLinux 9.x (Ceph host baseline) · Proxmox VE 9.x (cross-course integration) · Kubernetes 1.31+ (cross-course integration) · 2026-08-18

Why this matters in production

PG counts set by hand at pool creation go stale as pools grow and shrink. The autoscaler keeps them appropriate, and adopting it on a cluster with years of hand-tuned values needs care.

How it computes

The autoscaler targets roughly 100 PGs per OSD across all pools, allocated in proportion to each pool’s share of the data:

ceph osd pool autoscale-status

POOL        SIZE  TARGET SIZE  RATE  RAW CAPACITY  RATIO  PG_NUM  NEW PG_NUM  MODE
rbd-vms     40T                3.0   600T          0.20   1024    2048        warn
rgw-data   120T                1.5   600T          0.30   512     4096        warn

NEW PG_NUM is the recommendation. In warn mode it is reported; in on mode it is applied.

Adopting it safely

# 1. start in warn on every pool
for p in $(ceph osd pool ls); do
  ceph osd pool set $p pg_autoscale_mode warn
done

# 2. read the recommendations
ceph osd pool autoscale-status

# 3. apply deliberately, one pool at a time, in a window
ceph config set osd osd_max_backfills 1
ceph osd pool set rbd-vms pg_num 2048
# wait for active+clean

# 4. once pools are near target, switch to on
ceph osd pool set rbd-vms pg_autoscale_mode on

Guiding it with target_size_ratio

The autoscaler sizes pools by current data, which under-provisions a pool that is about to grow:

ceph osd pool set rgw-data target_size_ratio 0.6
ceph osd pool set rbd-vms target_size_ratio 0.3
ceph osd pool set backups target_size_ratio 0.1

Ratios express the expected eventual share of the cluster. The autoscaler then sizes for the expected state rather than the current one, which avoids a series of small splits as a pool fills.

target_size_bytes sets an absolute expectation instead. Use one or the other, not both.

Why the autoscaler is conservative about lowering pg_num

Raising pg_num splits PGs, which is metadata work followed by data movement as pgp_num follows. Lowering it merges PGs, which is the same work in reverse plus substantially more data movement as PGs consolidate.

The autoscaler therefore applies a threshold before recommending a decrease — it will not shrink a pool for a small overshoot, because the cost of the merge exceeds the benefit of a marginally better count.

The operational reading is that a pool sitting somewhat above its ideal PG count is usually fine and left alone deliberately. A pool far below is the case worth acting on, because too few PGs costs distribution evenness, balancing granularity, and recovery parallelism.

If you find yourself wanting to shrink a pool substantially, the question worth asking first is whether the original pg_num was set for a cluster that has since shrunk — and whether adding OSDs is a better answer than merging PGs.

When to override

A pool about to grow substantially. Set target_size_ratio rather than overriding.
A pool with an unusual object size profile. Very large objects mean fewer objects per PG, which the autoscaler does not model directly.
A cluster near its PG-per-OSD ceiling. The autoscaler works within the total; if the total is the problem, fewer pools is the answer.

Quiz

Knowledge check · 4 questions

Q1. Why should pg_autoscale_mode be set to warn before on when adopting the autoscaler on an established cluster?

Because on mode requires the balancer to be disabled, and re-enabling it afterwards restarts the movement Because the first recommendation can differ substantially and on mode would apply it at a time nobody chose Because warn mode gathers usage across several intervals and so produces a more accurate recommendation Because on mode refuses to change pg_num on a pool that already holds data, so it must start empty
Q2. The autoscaler applies a threshold before recommending a decrease in pg_num.

True False
Q3. A pool is expected to grow from 5 TB to 200 TB over the next year. The autoscaler recommends 128 PGs. Advise.

New RGW data pool currently holding 5 TB, expected to reach 200 TB within a year as an archival migration proceeds. Cluster has 96 OSDs. The autoscaler in warn mode recommends pg_num 128 based on the current 5 TB. Other pools hold 300 TB combined.
Q4. Describe the safe adoption path for the autoscaler on an established cluster.

Passing score: 75%. Answers are checked in this browser.

Production discipline

Adopt the autoscaler in warn mode on an established cluster, apply the large corrections yourself in a chosen window, and switch to on only once pools are near target — from there its incremental adjustments are exactly what you want. Use target_size_ratio to tell it about expected growth rather than overriding recommendations, and apply a large pg_num while a pool is still small, since splitting is cheap when empty and merging is never cheap.

Cross-course references

Ceph: Part XXI (PG Autoscale) for the module in operation.
Ceph: Part XVIII (Placement Groups) for what pg_num controls.
Ceph: Part X (Manager Daemons) for the module alongside the balancer.