Why RBD generally wants replication

Why this matters in production

RBD is the most-deployed Ceph interface and the one where the pool choice has the most visible consequences. Getting it right at provisioning time costs nothing; getting it wrong costs a migration of every image.

What a block device actually does

An RBD image is a flat address space carved into objects — 4 MiB by default. The guest filesystem and application write into it with whatever pattern they have, and the common ones are all small:

Only the last row is EC-friendly, and it is the least common in steady state.

Why replication fits

• Small random writes are cheap. Three writes, no read phase, one device-latency term.
• Latency is bounded by three OSDs, not k+m.
• Overwrites are native — no read-modify-write path exists.
• Degraded reads stay fast, served from the primary at normal speed.

The cost is 3× capacity, and for the volumes that carry a workload’s working set that is usually the right purchase.

The RBD exceptions

EC-backed RBD is a good fit when the write pattern is large and sequential:

• Backup target volumes — an agent streaming multi-megabyte blocks
• Media and archive volumes — written once, read occasionally
• Container image layer storage — write-once, medium-sized objects
• Cold VM images — powered-off templates and snapshots

The test is not “is it RBD” but “does it write large aligned blocks.”

Structuring the choice

# replicated: the default class
ceph osd pool create rbd-standard replicated
ceph osd pool application enable rbd-standard rbd

# EC-backed: for bulk volumes
ceph osd pool create rbd-bulk-meta 32 32 replicated
ceph osd pool create rbd-bulk-data erasure ec42
ceph osd pool set rbd-bulk-data allow_ec_overwrites true
ceph osd pool application enable rbd-bulk-meta rbd
ceph osd pool application enable rbd-bulk-data rbd

rbd create --size 20T --data-pool rbd-bulk-data rbd-bulk-meta/backup-target-01

Expose both as distinct StorageClasses or provisioning options so the choice is made once, explicitly, at creation.

Quiz

Production discipline

Make the write-pattern criterion explicit in the storage catalogue so consumers can self-select, and audit placements periodically against measured write sizes rather than against what teams declared at provisioning. Where a volume is on the wrong class, plan the migration during a window rather than leaving a known mismatch in place.

Cross-course references

• Kubernetes: StorageClass selection is exactly where this decision belongs
• Linux: choosing a RAID level per LUN according to workload is the same practice