Respond to Ceph credential compromise

This runbook covers Respond to Ceph credential compromise on a Ceph Tentacle cluster.

When to use

• Active production scenario matching the impact (security-relevant)
• The prerequisites are met

Prerequisites

Before starting this runbook, verify each prerequisite has been met. If a prerequisite is not met, resolve it first or escalate.

• A cephx key is suspected of compromise
• A client key is suspected of compromise

Pre-checks

Collect evidence before taking action. Evidence is the proof the runbook worked.

# The key is identified
# The compromised key is enumerated: ceph auth list
# Recent activity of the key is captured: ceph log last
# The cluster is HEALTH_OK

Procedure

Step 1

Disable the compromised key: ceph auth rm client.<user>

Step 2

Create a new key: ceph auth get-or-create-key client.<user>

Step 3

Distribute the new key to the legitimate consumers

Step 4

Disable any access from the compromised key

Step 5

Log the incident

Step 6

Audit other credentials

Verification

After completing the procedure, verify the result:

# The compromised key is disabled
# The legitimate key is replaced
# Cluster is HEALTH_OK
# The incident is documented

Rollback

If the procedure does not produce the expected verification, roll back:

• If the same client has multiple keys: rotate all of them
• If the root cause is unknown: full credential rotation
• If the cluster is being attacked: take it offline temporarily

Escalation

If you cannot roll back or the rollback fails, escalate:

• Security team
• Storage team
• Disaster recovery team