Running a capability audit

Why this matters in production

Capabilities accumulate. Entities are created for projects that end, grants are widened during incidents and never narrowed, and credentials outlive the people who made them. An audit is how that entropy gets reversed, and having a procedure is what makes it happen more than once.

Step 1: enumerate everything

Three databases, not one:

# cephx
ceph auth ls -f json > audit/cephx.json

# RGW users
radosgw-admin user list > audit/rgw-users.json
for u in $(radosgw-admin user list | jq -r '.[]'); do
  radosgw-admin user info --uid="$u"
done > audit/rgw-user-details.json

# dashboard users
ceph dashboard ac-user-show > audit/dashboard-users.json

Keep the outputs. They are both the audit baseline and the rollback source.

Step 2: identify the consumer of each entity

# who is actually connected
ceph daemon mon.$(hostname -s) sessions \
  | jq -r '.[].entity_name' | sort | uniq -c | sort -rn

# where keyrings exist
ansible all -m shell -a 'ls /etc/ceph/*.keyring 2>/dev/null'

An entity with no active sessions and no keyring anywhere on the fleet is a strong removal candidate. An entity with sessions from an unexpected host is a finding in its own right.

Step 3: assess each capability set

For each entity ask:

1. Does the consumer still exist?
2. Is every subsystem capability needed? (An RBD client needs no mds.)
3. Is every permission letter needed? (Does it need x, or only rw?)
4. Is it scoped to the pools it actually uses?
5. Would a maintained profile do the job better?

# a quick pass for the obvious problems
ceph auth ls | grep -B2 'allow \*'

Any allow * outside daemon entities and client.admin is a finding.

Step 4: reduce, with a rollback path

# capture before changing
ceph auth get client.example -o audit/rollback/client.example.keyring

# apply the reduced set — the complete intended set, not a delta
ceph auth caps client.example \
    mon 'profile rbd' \
    osd 'profile rbd pool=rbd-vms'

# verify positively and negatively
ceph -n client.example --keyring ... -s
rados -n client.example --keyring ... -p other-pool ls   # must fail

Change one entity at a time and confirm its consumer still works before moving on. ceph auth caps replaces the whole set, so the exported file is the only way back.

Step 5: cadence and triggers

The last row catches the most common source of drift: emergency grants that were never reversed.

Quiz

Production discipline

Script the enumeration and export steps so an audit is a command rather than a project, and sample monitor sessions continuously so the consumer map is always current rather than assembled under pressure. Add a capability review to your incident closure checklist — emergency grants that survive the incident are where most privilege drift originates.

Cross-course references

• Kubernetes: RBAC audits face the same challenge of distinguishing what exists from what is used
• Linux: periodic review of accounts, sudoers, and SSH keys is the same exercise with the same drift