The cephx authentication protocol

Intermediate⏱ ~17 minceph

What you'll learn

Describe the cephx authentication exchange
Explain what cephx does and does not protect
Verify that cephx is enabled cluster-wide
Diagnose common authentication failures

Prerequisites

None — start here.

Verified against Ceph Tentacle 20.2.x · Ceph Squid 19.2.x (supported previous) · cephadm matches the verified Ceph release · podman 4.x · csi-rbd and csi-cephfs current · RBD / CephFS / RGW current (matches Ceph release) · Linux kernel 5.15+ (5.10 minimum) · Ubuntu 24.04 LTS (Ceph host baseline) · Debian 12 (Bookworm) (Ceph host baseline) · Rocky Linux / RHEL / AlmaLinux 9.x (Ceph host baseline) · Proxmox VE 9.x (cross-course integration) · Kubernetes 1.31+ (cross-course integration) · 2026-08-18

Why this matters in production

cephx is on by default and mostly invisible, which means most operators first encounter it when something breaks. Understanding the exchange makes those failures readable, and understanding its boundaries prevents you from assuming protections it does not provide.

The exchange

sequenceDiagram
    participant C as Client
    participant M as Monitor
    participant O as OSD
    C->>M: I am client.rbd-prod
    M->>M: look up shared secret
    M-->>C: session key, encrypted with the shared secret
    C->>C: decrypt (proves it holds the secret)
    C->>M: request a ticket for OSDs
    M-->>C: ticket, signed and time-limited
    C->>O: request + ticket
    O->>O: verify the ticket signature
    O-->>C: response

The properties that matter:

Mutual. The client proves it holds the secret; the monitor proves it too, because only a holder could have encrypted a usable session key.
The secret never crosses the wire. Only material encrypted with it.
Tickets are time-limited, so a captured ticket has bounded value.
Every message is signed, so tampering is detected.

What cephx does not do

This is the part that gets assumed wrongly:

Protection	cephx
Authentication of clients and daemons	yes
Authorisation via capabilities	yes
Tamper detection on messages	yes
Encryption of data in transit	no — that is msgr2 `secure` mode
Encryption of data at rest	no — that is dm-crypt or SED
Protection from a stolen keyring file	no

cephx authenticates. It does not, by itself, make the wire confidential. A network observer with cephx alone can read object data in flight. Wire encryption is a separate setting.

Verifying it is on

ceph config get mon auth_cluster_required
ceph config get mon auth_service_required
ceph config get mon auth_client_required
# all three should be: cephx

auth_cluster_required covers daemon-to-daemon, auth_service_required covers daemons serving clients, auth_client_required covers what clients must present. Setting any to none disables authentication for that direction and should never appear in production.

Common failures

ceph auth get client.rbd-prod
ceph -n client.rbd-prod --keyring /etc/ceph/ceph.client.rbd-prod.keyring -s

Error	Cause
`EACCES: access denied`	key mismatch or entity does not exist
`authentication error (1) Operation not permitted`	wrong or missing keyring
`monclient: authenticate NOTE: no keyring found`	keyring file not at any searched path
Works then fails after some hours	clock skew — tickets are time-based

Quiz

Knowledge check · 4 questions

Q1. Does enabling cephx encrypt Ceph data in transit?

Yes — cephx negotiates a session key, and that key encrypts every subsequent message Only client traffic is encrypted; OSD-to-OSD replication is signed but left in clear No — cephx authenticates and signs messages; wire encryption requires msgr2 secure mode Only when auth_cluster_required is set to cephx, which encrypts daemon-to-daemon traffic
Q2. A client's secret key is transmitted to the monitor during authentication.

True False
Q3. Diagnose authentication that worked yesterday.

A monitoring client that has worked for months begins failing with authentication errors a few hours after its host was rebooted following a power event. The keyring file is unchanged, the entity exists, and other clients on other hosts work fine.
Q4. Why do monitors issue tickets rather than having OSDs authenticate clients directly?

Passing score: 75%. Answers are checked in this browser.

Production discipline

Verify all three auth_*_required settings are cephx as part of cluster commissioning and after any upgrade, and alert on clock skew as an authentication risk rather than only a monitor concern. Keep the search paths for keyring files consistent across hosts so a missing-keyring error means what it says.

Cross-course references

Kubernetes: ServiceAccount tokens are the same ticket pattern with the same expiry considerations
Linux: Kerberos ticket-granting is the direct ancestor of this design