Namespaces when pool-per-tenant stops scaling

Advanced⏱ ~18 mincephrbd

What you'll learn

Recognise when pool-per-tenant stops scaling
Implement namespace-scoped tenancy
Understand what namespaces do not isolate
Migrate a tenant between models

Prerequisites

None — start here.

Verified against Ceph Tentacle 20.2.x · Ceph Squid 19.2.x (supported previous) · cephadm matches the verified Ceph release · podman 4.x · csi-rbd and csi-cephfs current · RBD / CephFS / RGW current (matches Ceph release) · Linux kernel 5.15+ (5.10 minimum) · Ubuntu 24.04 LTS (Ceph host baseline) · Debian 12 (Bookworm) (Ceph host baseline) · Rocky Linux / RHEL / AlmaLinux 9.x (Ceph host baseline) · Proxmox VE 9.x (cross-course integration) · Kubernetes 1.31+ (cross-course integration) · 2026-08-18

Why this matters in production

Pool-per-tenant is the clearest isolation model and it has a hard ceiling in PG count that arrives sooner than most teams expect.

When pool-per-tenant stops scaling

ceph osd pool ls | wc -l
ceph pg stat
ceph osd df | awk 'NR>1 {print $NF}' | head -3

# PGs per OSD, the number that constrains pool count
ceph osd df --format json | python3 -c '
import sys,json
d = json.load(sys.stdin)["nodes"]
pgs = [o["pgs"] for o in d if o.get("pgs") is not None]
print("osds:", len(pgs), " mean PGs/OSD: %.0f" % (sum(pgs)/len(pgs)),
      " max:", max(pgs))'

Symptom	Meaning
`TOO_MANY_PGS` health warning	the PG budget is exhausted
Rising OSD memory	each PG has a fixed memory cost
Slow peering after restarts	PG count drives peering work
Every new tenant needs a smaller `pg_num`	the model is at its limit

Each pool needs enough PGs to distribute well, and every OSD has a
practical PG ceiling around 100–200. That product is the tenant ceiling.

Namespace-scoped tenancy

rbd namespace create shared-rbd/acme-prod
rbd namespace ls shared-rbd

ceph auth get-or-create client.acme-prod \
  mon 'profile rbd' \
  osd 'profile rbd pool=shared-rbd namespace=acme-prod' \
  mgr 'profile rbd pool=shared-rbd namespace=acme-prod'

rbd create --size 100G shared-rbd/acme-prod/vol01
rbd -n client.acme-prod ls shared-rbd/acme-prod

One pool, one PG budget, many tenants. The capability scoping is
enforced by the OSDs exactly as pool scoping is.

What namespaces do not isolate

Property	Pool per tenant	Namespace per tenant
Access control	yes	yes
Separate CRUSH placement	yes	no
Separate replication or EC profile	yes	no
Per-tenant quota	yes	no — quota is per pool
Separate PG budget	yes	no
Independent capacity accounting	yes	limited
Noisy-neighbour separation	partial	no

ceph osd pool get-quota shared-rbd

The quota gap is the significant one: namespaces share the pool's quota,
so per-tenant capacity limits must be enforced elsewhere — by the
provisioning layer, or by RBD image sizes.

# per-namespace usage, for accounting
rbd du shared-rbd/acme-prod 2>/dev/null | tail -1

Migrating between models

# pool to namespace: copy the images, then repoint the tenant
rbd migration prepare tenant-acme/vol01 shared-rbd/acme-prod/vol01
rbd migration execute shared-rbd/acme-prod/vol01
rbd migration commit shared-rbd/acme-prod/vol01

# update the capability to the new scope
ceph auth caps client.acme-prod \
  mon 'profile rbd' \
  osd 'profile rbd pool=shared-rbd namespace=acme-prod' \
  mgr 'profile rbd pool=shared-rbd namespace=acme-prod'

# verify, then remove the old pool
rbd -n client.acme-prod ls shared-rbd/acme-prod
ceph osd pool rm tenant-acme tenant-acme --yes-i-really-really-mean-it

Quiz

Knowledge check · 4 questions

Q1. What actually limits the number of pools in a pool-per-tenant model?

A hard pool count limit compiled into the monitors, which refuse pool creation beyond a fixed number The PG budget — each OSD has a practical PG ceiling, and each pool needs a minimum PG count to distribute evenly The monitor store, which grows with every pool definition until the OSDMap exceeds what a monitor can hold The CRUSH map, which gains a rule per pool and becomes too deep for the OSDs to evaluate efficiently
Q2. Under a namespace model, one tenant filling the pool makes every other tenant hit the limit at the same moment.

True False
Q3. Decide between pool and namespace tenancy.

A cluster has 60 tenant pools and is showing TOO_MANY_PGS. Twenty more tenants are planned. Tenants have identical durability requirements but individually billed capacity.
Q4. What does a namespace isolate, and what does it not?

Passing score: 75%. Answers are checked in this browser.

Production discipline

Move to namespace tenancy when the PG budget rather than the pool count becomes the constraint — but enforce per-tenant capacity in the provisioning layer, because namespaces share the pool’s quota. Keep pool-per-tenant where tenants genuinely need different placement or durability.

Cross-course references

Kubernetes: namespaces isolate names and RBAC, not node placement or quota by default
Linux: a scaling ceiling is usually a resource budget, not a stated limit