One identity per consumer

Why this matters in production

The cephx cap set is the only tenancy boundary Ceph has. A shared admin keyring on a hypervisor means any compromise of that hypervisor can delete every pool in the cluster, and the audit trail will show it as a legitimate admin operation.

The identity table

ceph auth get-or-create client.pve \
  mon 'profile rbd' \
  mgr 'profile rbd pool=rbd-vm' \
  osd 'profile rbd pool=rbd-vm'

ceph auth get-or-create client.csi-rbd-provisioner \
  mon 'profile rbd' mgr 'allow rw' osd 'profile rbd pool=rbd-k8s'

ceph fs authorize acme client.web /www rw
ceph auth get client.web

Profiles exist because hand-written caps go wrong. profile rbd grants exactly what an RBD client needs, including the object-map and lock operations that allow rw on its own does not cover.

Namespaces when a pool is too coarse

rbd namespace create rbd-k8s/tenant-a
rbd namespace ls rbd-k8s

ceph auth get-or-create client.tenant-a \
  mon 'profile rbd' \
  osd 'profile rbd pool=rbd-k8s namespace=tenant-a'

A pool per tenant burns PGs — 2400 PGs across the cluster does not divide into forty tenants. Namespaces are the right unit when the redundancy and device class are the same.

Auditing what exists

# entities holding wildcard capabilities
ceph auth ls -f json | python3 -c '
import sys,json
for e in json.load(sys.stdin)["auth_dump"]:
    caps = e.get("caps", {})
    if any("*" in v for v in caps.values()):
        print("%-34s %s" % (e["entity"], caps))'

client.admin                       {mon: allow *, osd: allow *, mds: allow *, mgr: allow *}
client.legacy-backup               {mon: allow r, osd: allow *}

The second line is the finding. osd 'allow *' on a backup account means the backup system can delete objects in every pool.

Rotation and revocation

# cephx has no second-key window: create a new entity, migrate, remove the old
ceph auth get-or-create client.pve2 \
  mon 'profile rbd' mgr 'profile rbd pool=rbd-vm' osd 'profile rbd pool=rbd-vm'
ceph auth get client.pve2 -o /etc/pve/priv/ceph/rbd-vm.keyring

# after every consumer is on the new identity
ceph auth rm client.pve
ceph auth ls | grep -c '^client\.'

# tighten in place, always writing the complete cap set
ceph auth caps client.backup \
  mon 'profile rbd-read-only' \
  osd 'profile rbd-read-only pool=rbd-vm'

Quiz

Production discipline

Give every consuming system its own entity, its own keyring and the narrowest profile that works — the cap set is the blast radius, and it is the only tenancy boundary Ceph enforces. Treat ceph auth caps as a whole-record write: read the existing caps first, edit them, apply the complete set.

Cross-course references

• Kubernetes: one ServiceAccount per workload, for the same reason
• Linux: a shared root credential turns every host compromise into a cluster compromise