Backing up and restoring the auth database

Why this matters in production

Restoring one accidentally deleted entity is a two-minute operation with a backup and a redistribution project without one.

Taking the backup

ceph auth export -o /secure/ceph-auth-$(date +%Y%m%d).export
chmod 600 /secure/ceph-auth-*.export

# per-entity, which restores more surgically
ceph auth ls --format json | python3 -c '
import sys,json
for e in json.load(sys.stdin)["auth_dump"]: print(e["entity"])' | \
while read ent; do
  ceph auth export "$ent" -o "/secure/entities/${ent}.export" 2>/dev/null
done
chmod 600 /secure/entities/*

Per-entity exports mean a single restore does not risk overwriting
entities that have legitimately changed since the backup.

head -3 /secure/entities/client.app-a.export

[client.app-a]
	key = AQ...
	caps mon = "profile rbd"
	caps osd = "profile rbd pool=app-a"

Storing it

An auth backup on the cluster it backs up protects against accidental
deletion and against nothing else.

Restoring a single entity

# what exists now
ceph auth get client.app-a 2>&1 | head -2

ceph auth import -i /secure/entities/client.app-a.export
ceph auth get client.app-a

Import restores the entity with its original key, which means existing
consumers holding that key work again immediately — no redistribution.

# verify the consumer recovered
IMAGE=vm-disk-01
rbd status app-a/${IMAGE} | grep -i watcher

Restoring the whole database

# this overwrites; understand what it reverts before running
ceph auth import -i /secure/ceph-auth-20260801.export

Importing a whole-database export adds and updates entities from the
file. Entities created after the backup are not removed by it, and
entities modified since the backup are reverted to their backed-up
state.

# compare before importing
diff <(ceph auth ls --format json | python3 -c '
import sys,json
for e in sorted(json.load(sys.stdin)["auth_dump"], key=lambda x: x["entity"]):
    print(e["entity"], sorted(e.get("caps", {}).items()))') \
     /secure/auth-inventory-20260801.txt

Quiz

Production discipline

Take per-entity auth exports, not only a whole-database one — restoring one entity then recovers its original key without reverting anything that legitimately changed. Store the export off the cluster and encrypted; it contains every key.

Cross-course references

• Kubernetes: restoring one Secret differs from restoring an etcd snapshot
• Linux: granular backups make recovery surgical rather than wholesale