Skip to main content
RunBook Academy

CephXXXIV · Multi-Tenancy ConceptsMulti-Tenancy Concepts

Matching the isolation mechanism to the tenant

Advanced⏱ ~17 minceph

What you'll learn

  • Enumerate the isolation mechanisms and their costs
  • Match a mechanism to a stated isolation requirement
  • Combine mechanisms where appropriate
  • Recognise requirements no mechanism satisfies

Prerequisites

None — start here.

Verified against Ceph Tentacle 20.2.x · Ceph Squid 19.2.x (supported previous) · cephadm matches the verified Ceph release · podman 4.x · csi-rbd and csi-cephfs current · RBD / CephFS / RGW current (matches Ceph release) · Linux kernel 5.15+ (5.10 minimum) · Ubuntu 24.04 LTS (Ceph host baseline) · Debian 12 (Bookworm) (Ceph host baseline) · Rocky Linux / RHEL / AlmaLinux 9.x (Ceph host baseline) · Proxmox VE 9.x (cross-course integration) · Kubernetes 1.31+ (cross-course integration) · 2026-08-18

Not yet marked complete on this device.

Why this matters in production

Multi-tenancy discussions default to the strongest available separation, which is usually more expensive than the requirement justifies. Matching the mechanism to what is actually required — per tenant, not per platform — is what makes a shared cluster economical.

The ladder

From cheapest to most complete:

LevelMechanismIsolatesCost
1Capability scope (namespace)data accessnone
2Capability scope (pool)data access, properties, accountingPGs
3CephFS subvolumedirectory, quotanegligible
4RGW tenantbucket namespace, quotanone
5Separate CephFS filesystemMDS resources, metadataMDS daemons
6Device-class-separated poolsphysical deviceshardware
7Separate clustereverythinga whole cluster

Levels 1–4 are effectively free and should be the default. Level 5 and 6 are for genuine resource-contention problems. Level 7 is for requirements that shared infrastructure cannot meet.

Matching requirement to level

RequirementMinimum level
“Tenants must not read each other’s data”1
“Each tenant needs its own quota”2 (RBD), 3 (CephFS), 4 (RGW)
“Tenants need different durability”2
“We bill by consumption”2
“One tenant must not slow others”6
“Tenant data must be on separate hardware”6
“Regulatory separation of infrastructure”7

Note that “must not read each other’s data” — the requirement people usually mean by isolation — is satisfied at level 1, the cheapest option.

Combining

Mechanisms compose. A common production arrangement:

# device-class-separated tiers
ceph osd crush rule create-replicated nvme-rule default host nvme
ceph osd pool create rbd-gold; ceph osd pool set rbd-gold crush_rule nvme-rule
ceph osd pool create rbd-standard

# tenants as namespaces inside each tier
rbd namespace create rbd-gold/tenant-a
rbd namespace create rbd-standard/tenant-b

Tiers provide hardware separation where it is needed; namespaces provide tenant separation everywhere at no PG cost.

What no mechanism provides

Performance isolation between tenants sharing devices. A tenant saturating an OSD affects everyone whose data is on it. Ceph has no per-tenant I/O quota. If one tenant must be unable to affect another, the answer is separate hardware — level 6 at minimum.

Quiz

Knowledge check · 4 questions

  1. Q1. A requirement states that tenants must not be able to read each other's data. What is the minimum mechanism that satisfies it?

  2. Q2. Ceph provides per-tenant I/O quotas so one tenant cannot affect another's performance.

  3. Q3. Clarify an isolation requirement before designing for it.

    A regulated financial customer states that their data "must be fully isolated from other tenants". The platform team is preparing to build them a dedicated cluster at significant cost.

  4. Q4. Why does Ceph not provide per-tenant I/O scheduling?

Passing score: 75%. Answers are checked in this browser.

Production discipline

Resolve what “isolation” means before designing for it — the word covers three requirements with order-of-magnitude cost differences, and the expensive interpretation is the one people assume. Default to the cheapest mechanism that meets the confirmed requirement and record which interpretation was agreed.

Cross-course references

  • Kubernetes: namespace versus node pool versus separate cluster is the identical ladder
  • Linux: containers versus VMs versus separate machines follows the same reasoning