Volume encryption with Ceph-CSI

Advanced⏱ ~18 minkubectlrbdcryptsetup

What you'll learn

Configure CSI-layer volume encryption
Understand what it protects against
Manage the encryption keys
Recognise the operational consequences

Prerequisites

None — start here.

Verified against Ceph Tentacle 20.2.x · Ceph Squid 19.2.x (supported previous) · cephadm matches the verified Ceph release · podman 4.x · csi-rbd and csi-cephfs current · RBD / CephFS / RGW current (matches Ceph release) · Linux kernel 5.15+ (5.10 minimum) · Ubuntu 24.04 LTS (Ceph host baseline) · Debian 12 (Bookworm) (Ceph host baseline) · Rocky Linux / RHEL / AlmaLinux 9.x (Ceph host baseline) · Proxmox VE 9.x (cross-course integration) · Kubernetes 1.31+ (cross-course integration) · 2026-08-18

Why this matters in production

CSI-layer encryption puts LUKS between the pod and the RBD image, which protects a specific set of things and complicates several operations.

Configuring it

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: ceph-rbd-encrypted
provisioner: rbd.csi.ceph.com
parameters:
  clusterID: b3d5f2a1-...
  pool: k8s-rbd
  encrypted: "true"
  encryptionKMSID: "kms-config-id"
  csi.storage.k8s.io/provisioner-secret-name: csi-rbd-secret
  csi.storage.k8s.io/provisioner-secret-namespace: ceph-csi
  csi.storage.k8s.io/node-stage-secret-name: csi-rbd-secret
  csi.storage.k8s.io/node-stage-secret-namespace: ceph-csi

# KMS configuration, in the ceph-csi-encryption-kms-config ConfigMap
{
  "kms-config-id": {
    "encryptionKMSType": "vault",
    "vaultAddress": "https://vault.example.com",
    "vaultAuthPath": "/v1/auth/kubernetes/login",
    "vaultRole": "csi-kubernetes",
    "vaultBackendPath": "secret/",
    "vaultTLSServerName": "vault.example.com"
  }
}

What it protects

Threat	Protected?
Someone reading the raw device on an OSD host	yes
A stolen or decommissioned drive	yes
An operator with cluster admin reading the pool	yes
Data in transit between client and OSD	no — use msgr2 `secure`
A compromised node running the pod	no — the volume is decrypted there
A compromised KMS	no

The distinction that matters: the data is encrypted at rest in Ceph and decrypted on the node where the pod runs. Anyone with access to that node while the volume is mounted sees plaintext.

Key management

Per-volume keys, held by the KMS
  → the CSI node plugin fetches the key at stage time
  → opens the LUKS device
  → the pod sees a decrypted block device

KMS option	Consideration
Kubernetes Secrets	simple; keys are in etcd
HashiCorp Vault	proper key management; a dependency
AWS KMS, Azure Key Vault	managed; cloud-specific

# with Secrets-based KMS, the key is in the cluster
# SECRET is the KMS Secret named by the StorageClass `encryptionKMSID`;
# list candidates with `kubectl -n ceph-csi get secret`:
SECRET=ceph-csi-encryption-kms-config

kubectl -n ceph-csi get secret "$SECRET" -o yaml

Secrets-based key storage means anyone who can read secrets in that namespace can decrypt the volumes, which limits what the encryption protects against.

Operational consequences

Operation	Effect of encryption
Provisioning	slower; LUKS format on first use
Snapshots	the snapshot is of the encrypted image
Cloning	clone inherits the encryption and the key reference
Expansion	requires the LUKS header to be resized too
Recovery from a Ceph-side copy	impossible without the key
KMS unavailable	volumes cannot be staged; pods cannot start
Key lost	data is unrecoverable

# a volume's encryption status
IMAGE=vm-disk-01
rbd info k8s-rbd/${IMAGE} | grep -i meta

Quiz

Knowledge check · 4 questions

Q1. What does CSI-layer encryption protect against that OSD-level dm-crypt does not?

Data in transit between the client and the OSDs, which dm-crypt leaves in plaintext once it is on the wire A compromised or untrusted storage layer — the cluster stores ciphertext it cannot read, since the key is outside Ceph A compromised application node, since the key never leaves the KMS and the pod is handed blocks that are already decrypted Key loss, because the CSI layer escrows a recovery key in the spare slots of the volume's LUKS header
Q2. An encrypted volume can be recovered from a Ceph-side copy if the key is lost.

True False
Q3. Evaluate a proposal for CSI-layer encryption.

A team proposes enabling CSI encryption on all volumes for compliance, using Kubernetes Secrets as the key store.
Q4. Why does an encrypted volume expansion need more than resizing the RBD image?

Passing score: 75%. Answers are checked in this browser.

Production discipline

Give the KMS the same availability as the Ceph cluster — the node plugin fetches keys at stage time, so a KMS outage prevents every encrypted volume from mounting including during an unrelated incident. Establish key backup before enabling encryption; key loss makes the data unrecoverable regardless of replica count.

Cross-course references

Kubernetes: any external dependency in the storage path inherits the storage availability requirement
Linux: LUKS key management determines what disk encryption actually protects