Encryption at rest with dm-crypt and self-encrypting drives

Why this matters in production

At-rest encryption is frequently a regulatory requirement and frequently misunderstood. It protects against one specific threat — physical possession of the device — and against nothing else. Deploying it without understanding that boundary produces a checkbox rather than a control.

What it protects against

The running system has the volume unlocked, so anything with access to that host reads plaintext. At-rest encryption is about the device leaving your control, and that is the whole of it.

Deploying encrypted OSDs

ceph orch apply osd --all-available-devices --encrypted

Or in a service specification:

service_type: osd
service_id: encrypted-hdd
placement:
  host_pattern: 'ceph-osd-*'
spec:
  data_devices:
    rotational: 1
  encrypted: true

cephadm creates a LUKS container per device and stores the dm-crypt key in the monitor’s config-key store, retrieving it at OSD startup.

MAPPER_DEVICE=/dev/sdb
ceph config-key ls | grep dm-crypt
lsblk
cryptsetup status ${MAPPER_DEVICE}

Encryption must be chosen at OSD creation. Converting an existing OSD means destroying and re-creating it, so a fleet-wide conversion is a rolling rebuild.

Self-encrypting drives

SEDs encrypt in the drive controller with no host CPU cost, managed through TCG Opal or the SAS security commands.

SEDs are attractive for their crypto-erase capability — a decommission becomes a key destruction rather than a wipe — but they place your trust in drive firmware, and firmware implementation flaws in SED products have been found repeatedly. dm-crypt is the more conservative choice.

Key management is the real problem

By default cephadm stores dm-crypt keys in the monitor config-key store — which means the keys live in the cluster whose data they protect.

That is fine for the drive-removal threat: a drive taken out of the chassis is useless without the monitors. It is not sufficient if the threat model includes taking the whole cluster, and it creates a hard dependency: lose the monitor store and every OSD is unrecoverable.

# back up the config-key store along with the monitor store
ceph config-key dump > /secure-backup/config-key-dump.json

Quiz

Production discipline

State explicitly, in the security documentation, that at-rest encryption addresses device loss and nothing else — the checkbox invites a broader assumption that will otherwise go unchallenged. Where encrypted OSDs are used, treat monitor store and config-key backups as protecting the data itself, store them outside the cluster, and test the restore.

Cross-course references

• Kubernetes: encrypted etcd with keys stored in the same cluster has the identical circular dependency
• Linux: LUKS key management and the running-system caveat are the same in any deployment