A threat model for a Ceph cluster

Advanced⏱ ~18 minceph

What you'll learn

Enumerate the attack surface of a Ceph cluster
Map each control to the position it defends
Identify the gaps a default deployment leaves
Prioritise hardening work by what it actually prevents

Prerequisites

None — start here.

Verified against Ceph Tentacle 20.2.x · Ceph Squid 19.2.x (supported previous) · cephadm matches the verified Ceph release · podman 4.x · csi-rbd and csi-cephfs current · RBD / CephFS / RGW current (matches Ceph release) · Linux kernel 5.15+ (5.10 minimum) · Ubuntu 24.04 LTS (Ceph host baseline) · Debian 12 (Bookworm) (Ceph host baseline) · Rocky Linux / RHEL / AlmaLinux 9.x (Ceph host baseline) · Proxmox VE 9.x (cross-course integration) · Kubernetes 1.31+ (cross-course integration) · 2026-08-18

Why this matters in production

Hardening without a threat model produces effort spent on controls that stop nothing while the actual exposure remains.

Attacker positions

Position	Reaches
On the public network	monitor and OSD ports, RGW, dashboard
On the cluster network	OSD replication traffic
With a client keyring	whatever its capabilities permit
With `client.admin`	everything
On a Ceph host, unprivileged	local keyrings if readable
On a Ceph host, root	every key on that host, every device
With physical device access	BlueStore data at rest

The order matters: most real compromises start at one of the first
three, not the last.

Which control defends which position

Control	Defends against
cephx authentication	an unauthenticated party on the public network
Capability scoping	a compromised client exceeding its purpose
msgr2 secure mode	passive interception and message tampering on the wire
Network segmentation	reaching cluster ports from outside
Host hardening	pivoting from a host to the cluster
Keyring file permissions	unprivileged local users reading keys
Encryption at rest	physical device theft or improper disposal

Each control has exactly one job. Encryption at rest does nothing about a
leaked keyring; capability scoping does nothing about a stolen disk.

What a default deployment leaves open

ceph config get global auth_cluster_required
ceph config get global auth_service_required
ceph config get global auth_client_required

cephx is on by default. That is the significant default that is correct.

Default	Status
cephx enabled	on — good
msgr2 available	on — but `crc` mode, not `secure`
Cluster network separated	only if configured
Firewall restricting Ceph ports	not by Ceph’s doing
Capabilities minimal	only if written that way
Keys rotated	never, unless done
Encryption at rest	only if OSDs were created encrypted

ceph config get global ms_cluster_mode
ceph config get global ms_service_mode
ceph config get global ms_client_mode

crc mode authenticates the connection and detects corruption. It does
not encrypt. Traffic is readable to anyone on the path.

Prioritising

Highest value first, for a typical cluster:
  1. network segmentation and firewalling — closes the largest surface
  2. capability scoping — bounds the damage of any client compromise
  3. keyring handling and permissions — the most common real leak
  4. msgr2 secure mode — closes on-the-wire exposure
  5. host hardening — reduces pivot paths
  6. encryption at rest — addresses physical and disposal risk

Encryption at rest is often done first because it is the most visible to
auditors, and it is the control that addresses the least likely threat
for a cluster in a controlled facility.

Quiz

Knowledge check · 4 questions

Q1. What does msgr2 in its default `crc` mode provide?

Connection authentication and corruption detection, with message contents travelling in the clear Full encryption of message contents, using the cephx session key derived when the connection is established No protection at all beyond the TCP checksum, since crc names the checksum rather than any Ceph mechanism Encryption of OSD replication traffic only, leaving the client-facing public network readable to anyone on the path
Q2. A cluster whose OSDs are all encrypted at rest is no better protected against a leaked client keyring than one with no encryption at all.

True False
Q3. Prioritise hardening work for a new cluster.

A cluster is being hardened before release. An auditor has asked about encryption at rest. The cluster is in a controlled facility with no network segmentation and permissive client capabilities.
Q4. Why does a keyring leak outrank most other exposures?

Passing score: 75%. Answers are checked in this browser.

Production discipline

Map each control to the attacker position it defends before spending effort: encryption at rest does nothing about a leaked keyring, and capability scoping does nothing about a stolen disk. Note that msgr2’s default crc mode does not encrypt — secure must be set explicitly.

Cross-course references

Kubernetes: ServiceAccount tokens are bearer credentials with the same properties
Linux: controls defend specific positions; a threat model is what assigns them