Three ways to attach an RBD image

Advanced⏱ ~18 minrbdrbd-nbd

What you'll learn

Compare kernel RBD, librbd, and rbd-nbd
Match each to its appropriate use case
Understand the failure behaviour of each
Diagnose problems specific to each mode

Prerequisites

None — start here.

Verified against Ceph Tentacle 20.2.x · Ceph Squid 19.2.x (supported previous) · cephadm matches the verified Ceph release · podman 4.x · csi-rbd and csi-cephfs current · RBD / CephFS / RGW current (matches Ceph release) · Linux kernel 5.15+ (5.10 minimum) · Ubuntu 24.04 LTS (Ceph host baseline) · Debian 12 (Bookworm) (Ceph host baseline) · Rocky Linux / RHEL / AlmaLinux 9.x (Ceph host baseline) · Proxmox VE 9.x (cross-course integration) · Kubernetes 1.31+ (cross-course integration) · 2026-08-18

Why this matters in production

The three modes have genuinely different properties, and the differences matter most at exactly the wrong moment — when something fails. Choosing deliberately rather than by default is worth the few minutes it takes.

Kernel RBD

rbd map rbd-vms/data-vol
# /dev/rbd0
mkfs.xfs /dev/rbd0
mount /dev/rbd0 /mnt/data
rbd showmapped
rbd unmap /dev/rbd0

In-kernel, so the fastest path with no context switching to userspace.

Best performance, particularly for small random I/O
Fewest features — support depends on kernel version
Failure mode: a stuck RBD device can block processes in uninterruptible sleep and may require a reboot to clear
Upgrades require a kernel upgrade, meaning a reboot

librbd

Used in-process by QEMU/KVM and by applications linking librados.

<disk type='network' device='disk'>
  <source protocol='rbd' name='rbd-vms/vm-disk-01'>
    <host name='10.20.0.10' port='3300'/>
  </source>
</disk>

All features supported, always current with the Ceph release
Client-side caching configurable per image
Failure mode: contained within the process — a hung image affects that VM, not the host
Upgrades are a package update and a process restart

This is the default for virtualisation and the right choice there.

rbd-nbd

rbd-nbd map rbd-vms/data-vol
# /dev/nbd0
rbd-nbd list-mapped
rbd-nbd unmap /dev/nbd0

Userspace librbd exposed through the kernel’s NBD driver.

Full feature support on any kernel with NBD
Somewhat slower than kernel RBD due to the userspace round trip
Failure mode: the daemon can be killed, which cleanly releases the device — no reboot needed
Useful when you need modern features on an older kernel

Choosing

Use case	Mode
QEMU/KVM virtualisation	librbd
Kubernetes CSI (default)	kernel RBD
Bare-metal filesystem, modern kernel	kernel RBD
Bare-metal, older kernel, needs features	rbd-nbd
Anything requiring clean recovery from hangs	rbd-nbd or librbd

Diagnosing per mode

# kernel
rbd showmapped
dmesg | grep -i rbd
cat /sys/bus/rbd/devices/0/client_id

# nbd
rbd-nbd list-mapped
journalctl -u rbd-nbd@*

# librbd (via QEMU)
ceph daemon /var/run/ceph/ceph-client.*.asok perf dump

Quiz

Knowledge check · 4 questions

Q1. A host with a mapped kernel RBD device loses cluster connectivity while processes are writing. What is the likely outcome?

Processes receive I/O errors and carry on, because the filesystem remounts itself read-only Processes enter uninterruptible sleep, cannot be killed, and the host may need a reboot The device unmaps automatically once the kernel's request timeout expires The kernel serves reads from the page cache until the cluster becomes reachable again
Q2. rbd-nbd provides full RBD feature support regardless of kernel version.

True False
Q3. Choose an attachment mode for a critical application server.

A bare-metal application server with a long uptime requirement needs an RBD volume. The workload is moderate sequential I/O. The host runs an enterprise kernel that does not support object-map. Rebooting it requires a change window and a service failover.
Q4. Why is kernel RBD feature support a question about the specific host rather than about the Ceph version?

Passing score: 75%. Answers are checked in this browser.

Production discipline

Record which attachment mode each host uses and why, because the recovery procedure differs — the kernel-RBD hang path is the one that surprises people during a cluster outage. Test kernel feature support on the actual hosts rather than inferring it from documentation, since distribution backports make version numbers unreliable.

Cross-course references

Kubernetes: the CSI driver chooses kernel RBD or rbd-nbd, and the choice has these same consequences
Linux: in-kernel versus FUSE filesystems present the identical performance-versus-recoverability trade