client.admin and the discipline around it

Why this matters in production

client.admin can delete every pool in the cluster with one command. It is created at bootstrap, copied to convenient places, and used for everything because it always works. Narrowing its use is one of the highest-value security improvements available on a typical cluster and it costs almost nothing.

What it permits

ceph auth get client.admin
# caps: [mds] allow *
# caps: [mgr] allow *
# caps: [mon] allow *
# caps: [osd] allow *

Unrestricted across every subsystem: create and delete pools, change capabilities, remove OSDs, modify CRUSH, read and write every object. There is no operation it cannot perform.

Scoped operator credentials

Most routine work needs far less. Create credentials matching actual roles:

# read-only operations engineer
ceph auth get-or-create client.ops-readonly \
    mon 'allow r' mgr 'allow r' osd 'allow r'

# day-to-day operator: cluster state and OSD lifecycle, no pool deletion
ceph auth get-or-create client.ops \
    mon 'allow r, allow command "osd out", allow command "osd in", allow command "osd set", allow command "osd unset"' \
    mgr 'allow r' \
    osd 'allow r'

Command-level monitor capabilities are the mechanism for granting specific administrative actions without granting all of them. Verify each grant works and that the excluded commands are denied.

Controlling where the keyring lives

# find every copy
ansible all -m shell -a 'ls -l /etc/ceph/ceph.client.admin.keyring 2>/dev/null'

It belongs on the machines that genuinely need it — typically the monitors and a designated administrative host — and nowhere else. Every additional copy is another host whose compromise is a full cluster compromise.

With cephadm, administrative commands run through cephadm shell on a host with the admin label, which concentrates the credential rather than spreading it:

ceph orch host label add ceph-mon-01 _admin
ceph orch host label rm ceph-osd-07 _admin

Audit trail

ceph config set mon mon_cluster_log_to_file true
ceph log last 100 audit

The monitor audit log records administrative commands with the entity that issued them. This is only useful if entities are distinguishable — if everyone uses client.admin, the log tells you a command was run and nothing about who ran it.

Quiz

Production discipline

Restrict client.admin to the monitors and a designated administrative host, and treat its use as break-glass with a record of each occasion. Build per-role or per-person operator credentials so the monitor audit log answers the attribution question, and verify the excluded commands are actually denied rather than assuming the capability string is correct.

Cross-course references

• Kubernetes: cluster-admin used routinely creates the identical attribution and blast-radius problem
• Linux: shared root accounts versus per-user sudo is exactly this trade-off