CephFS quotas and their enforcement model

Intermediate⏱ ~16 minsetfattrgetfattr

What you'll learn

Set byte and file-count quotas on directories
Explain the client-side enforcement model
Quantify the expected overshoot
Choose quotas versus other capacity controls

Prerequisites

None — start here.

Verified against Ceph Tentacle 20.2.x · Ceph Squid 19.2.x (supported previous) · cephadm matches the verified Ceph release · podman 4.x · csi-rbd and csi-cephfs current · RBD / CephFS / RGW current (matches Ceph release) · Linux kernel 5.15+ (5.10 minimum) · Ubuntu 24.04 LTS (Ceph host baseline) · Debian 12 (Bookworm) (Ceph host baseline) · Rocky Linux / RHEL / AlmaLinux 9.x (Ceph host baseline) · Proxmox VE 9.x (cross-course integration) · Kubernetes 1.31+ (cross-course integration) · 2026-08-18

Why this matters in production

CephFS quotas are useful and are not a hard boundary. Deploying them with an accurate understanding of the overshoot avoids both the surprise of a tenant exceeding its limit and the mistake of relying on them as a security control.

Setting quotas

# byte quota
setfattr -n ceph.quota.max_bytes -v 10995116277760 /mnt/cephfs/projects/alpha

# file count quota
setfattr -n ceph.quota.max_files -v 5000000 /mnt/cephfs/projects/alpha

getfattr -n ceph.quota.max_bytes /mnt/cephfs/projects/alpha

Through the subvolume interface:

ceph fs subvolume resize cephfs project-alpha 10995116277760 --group_name research
ceph fs subvolume info cephfs project-alpha --group_name research

Setting a quota to 0 removes it.

How enforcement works

Quotas are enforced by the client, not by the MDS or the OSDs:

1. client writes into a quota-bounded tree
2. client periodically learns the tree's current usage from the MDS
3. when its local view shows the quota exceeded, it refuses further writes

Two consequences follow directly:

There is a propagation delay. Usage statistics reach the client after the fact, so a tenant writing quickly exceeds the limit before its client notices.
It depends on the client cooperating. A client that ignores the attribute is not stopped by anything.

The overshoot

overshoot ≈ write rate × statistics propagation interval × number of clients

A tenant with ten clients each writing 200 MB/s, with a several-second propagation interval, can exceed its quota by several gigabytes before any of them refuses a write. Plan capacity with that headroom included.

What quotas are and are not for

Use	Suitable?
Capacity management between cooperative tenants	yes
Preventing one team filling the filesystem	yes, with headroom
Enforcing a contractual limit exactly	no
Security boundary against a hostile tenant	no
Triggering alerts before capacity is exhausted	yes

For a security boundary, use path-restricted capabilities — enforced by the MDS server-side — and separate pools where the isolation must be absolute.

Monitoring

getfattr -n ceph.dir.rbytes /mnt/cephfs/projects/alpha
getfattr -n ceph.dir.rfiles /mnt/cephfs/projects/alpha

The recursive statistics attributes give current usage, which is what to alert on at, say, 80% of quota.

Quiz

Knowledge check · 4 questions

Q1. Where is a CephFS quota enforced?

In the OSDs, which reject the write once the pool's recorded quota is reached In the MDS, which checks every ancestor directory's usage on each write In the client, which learns usage from the MDS and refuses writes past the limit In the monitors, which track directory usage and revoke the client's capabilities
Q2. A CephFS quota is a suitable security boundary against a tenant that may run modified client software.

True False
Q3. Plan quotas for a shared research filesystem.

A CephFS filesystem will be shared by twelve research groups with contractual capacity allocations. Some groups run high-throughput data acquisition writing hundreds of megabytes per second from multiple clients simultaneously.
Q4. Why can CephFS quotas not be enforced server-side?

Passing score: 75%. Answers are checked in this browser.

Production discipline

Verify quota enforcement by actually exceeding a test quota from every client type in your fleet; support varies and a quota that silently does nothing looks identical to one that works. Include the expected aggregate overshoot in the capacity plan, and use path restrictions rather than quotas wherever a real boundary is required.

Cross-course references

Kubernetes: ResourceQuota admission has similar eventual-consistency behaviour
Linux: NFS quotas share the client-cooperation characteristic and its consequences