Operating CephFS subvolume tenancy over time

Why this matters in production

Subvolumes are easy to create and the operational questions arrive afterwards: growth, snapshots, and what happens at the limit.

Lifecycle after creation

ceph fs subvolume ls cephfs
ceph fs subvolume info cephfs acme-prod --group_name tenants

{
    "bytes_quota": 5497558138880,
    "bytes_used": 3211698176,
    "path": "/volumes/tenants/acme-prod/<uuid>",
    "mode": 16877,
    "created_at": "2026-03-14 09:12:01"
}

# per-tenant usage across a group
ceph fs subvolume ls cephfs --group_name tenants --format json | python3 -c '
import sys,json,subprocess
for s in json.load(sys.stdin):
    n = s["name"]
    i = json.loads(subprocess.check_output(
        ["ceph","fs","subvolume","info","cephfs",n,"--group_name","tenants",
         "--format","json"]))
    q = i.get("bytes_quota") or 0
    u = i.get("bytes_used") or 0
    pct = (100.0*u/q) if isinstance(q,int) and q else 0
    print("%-20s %8.1f GiB / %8.1f GiB  %5.1f%%" %
          (n, u/1024**3, (q/1024**3) if isinstance(q,int) else 0, pct))'

Changing a quota

ceph fs subvolume resize cephfs acme-prod 10995116277760 --group_name tenants

# removing the quota entirely
ceph fs subvolume resize cephfs acme-prod infinite --group_name tenants

Decreasing below current usage does not delete anything. It stops
growth, which is usually what is wanted, and it does surprise a tenant
who expected an error at set time rather than at write time.

Snapshots per tenant

ceph fs subvolume snapshot create cephfs acme-prod snap-$(date +%Y%m%d) \
  --group_name tenants
ceph fs subvolume snapshot ls cephfs acme-prod --group_name tenants

ceph fs subvolume snapshot info cephfs acme-prod snap-20260818 \
  --group_name tenants

ceph fs subvolume snapshot rm cephfs acme-prod snap-20260701 --group_name tenants

Snapshot capacity is not free and is not counted against the subvolume
quota — it consumes pool capacity as the snapshotted data diverges.

ceph df detail

Where quota enforcement happens

CephFS quotas are enforced by the client, not by the MDS or the OSDs.

ceph tell mds.0 client ls --format json | python3 -c '
import sys,json
for c in json.load(sys.stdin):
    md = c.get("client_metadata", {})
    print("%-18s %-14s %s" % (md.get("entity_id"), md.get("ceph_version","?")[:12],
                              md.get("hostname")))'

So a subvolume quota bounds a cooperating tenant. A pool quota, enforced
at the cluster, bounds any tenant.

Quiz

Production discipline

Use a per-tenant pool with a pool quota where the boundary must hold regardless of client behaviour — CephFS subvolume quotas are enforced by the client. Track snapshot retention separately; snapshots consume pool capacity without counting against the subvolume quota.

Cross-course references

• Kubernetes: ResourceQuota is enforced by the API server, not the workload
• Linux: a limit the client enforces bounds cooperation, not capability