RBD StorageClass parameters in depth

Advanced⏱ ~18 minkubectlrbd

What you'll learn

Configure the full parameter set deliberately
Choose a mounter appropriate to the environment
Recognise problematic parameter combinations
Tune per-workload where it matters

Prerequisites

None — start here.

Verified against Ceph Tentacle 20.2.x · Ceph Squid 19.2.x (supported previous) · cephadm matches the verified Ceph release · podman 4.x · csi-rbd and csi-cephfs current · RBD / CephFS / RGW current (matches Ceph release) · Linux kernel 5.15+ (5.10 minimum) · Ubuntu 24.04 LTS (Ceph host baseline) · Debian 12 (Bookworm) (Ceph host baseline) · Rocky Linux / RHEL / AlmaLinux 9.x (Ceph host baseline) · Proxmox VE 9.x (cross-course integration) · Kubernetes 1.31+ (cross-course integration) · 2026-08-18

Why this matters in production

Several StorageClass parameters have non-obvious effects and a few combinations fail in ways that are hard to diagnose.

The full parameter set

parameters:
  clusterID: <fsid>
  pool: k8s-rbd
  dataPool: k8s-rbd-ec              # EC data pool, replicated metadata
  imageFeatures: layering,exclusive-lock,object-map,fast-diff,deep-flatten
  mounter: rbd                       # or rbd-nbd
  csi.storage.k8s.io/fstype: ext4
  mkfsOptions: "-O ^has_journal"     # passed to mkfs
  mapOptions: "krbd:notrim"          # passed to rbd map
  unmapOptions: "force"
  tryOtherMounters: "true"
  encrypted: "false"
  thickProvision: "false"

The mounter

Mounter	Client	When
`rbd`	krbd, kernel	default; best performance
`rbd-nbd`	librbd via NBD	when the kernel lacks needed features

mounter: rbd-nbd
tryOtherMounters: "true"

tryOtherMounters falls back to the alternative if the preferred one fails, which handles a heterogeneous node fleet where some kernels support the image features and others do not.

# what the node kernel supports
uname -r
kubectl get nodes -o wide

Erasure coded data pools

parameters:
  pool: k8s-rbd-metadata      # replicated
  dataPool: k8s-rbd-ec        # erasure coded

RBD on EC requires a replicated pool for image metadata and an EC pool with allow_ec_overwrites for the data:

ceph osd pool set k8s-rbd-ec allow_ec_overwrites true

Omitting the replicated metadata pool is a configuration that simply does not work.

Problematic combinations

Combination	Problem
`mounter: rbd` with unsupported features	mapping fails on those nodes
EC `dataPool` without `allow_ec_overwrites`	writes fail
EC `dataPool` without a replicated `pool`	provisioning fails
`thickProvision: true` on a large volume	provisioning is very slow
`encrypted: true` without KMS configuration	staging fails
`fstype: xfs` with shrink expectations	xfs cannot shrink at all

# the CSI-provisioned image name, from `rbd ls k8s-rbd`:
IMAGE=csi-vol-8f3b1c2a-7d4e-11f0-9a3c-0242ac110002

# verify features against the node kernel
rbd feature disable "k8s-rbd/$IMAGE" object-map fast-diff

Per-workload tuning

# a database workload
parameters:
  pool: k8s-rbd-nvme
  csi.storage.k8s.io/fstype: xfs
  imageFeatures: layering,exclusive-lock,object-map,fast-diff,deep-flatten

# bulk data on erasure coding
parameters:
  pool: k8s-rbd-metadata
  dataPool: k8s-rbd-ec
  csi.storage.k8s.io/fstype: xfs

# the CSI-provisioned image name, from `rbd ls k8s-rbd`:
IMAGE=csi-vol-8f3b1c2a-7d4e-11f0-9a3c-0242ac110002

# per-image, after provisioning
rbd config image set "k8s-rbd/$IMAGE" rbd_qos_iops_limit 10000

Quiz

Knowledge check · 4 questions

Q1. Why does RBD on an erasure coded pool require a separate replicated metadata pool?

Erasure coding cannot store an object smaller than a full stripe, so the image header and the object map would have nowhere at all to live in such a pool Replication reads faster, and the header is read on every map operation while the data blocks are read only when touched The CSI driver requires two pools for every image so that provisioning and node staging can use separate credentials The image header and object map are updated with small partial writes, which EC handles worst — each becomes a read-modify-write across the stripe
Q2. One krbd StorageClass can map successfully on some nodes of a cluster and fail on others.

True False
Q3. Configure RBD on erasure coding for bulk data.

A team wants to use erasure coding for a large bulk-data PVC to save capacity. They have created an EC pool and set it as the StorageClass pool parameter.
Q4. What does `tryOtherMounters` solve?

Passing score: 75%. Answers are checked in this browser.

Production discipline

Set tryOtherMounters: "true" on any cluster with mixed kernel versions — krbd feature support varies and the failures appear as pods starting on some nodes and not others. For RBD on erasure coding, set both pool (replicated, for metadata) and dataPool (EC, with allow_ec_overwrites).

Cross-course references

Kubernetes: heterogeneous node capabilities need explicit fallback handling
Linux: metadata and bulk data warranting different storage is a general principle