rbd-nbd and when it is the right choice

Intermediate⏱ ~16 minrbd-nbdrbd

What you'll learn

Explain how rbd-nbd works
Identify the cases where it is the right client
Deploy and manage it
Understand its failure modes

Prerequisites

None — start here.

Verified against Ceph Tentacle 20.2.x · Ceph Squid 19.2.x (supported previous) · cephadm matches the verified Ceph release · podman 4.x · csi-rbd and csi-cephfs current · RBD / CephFS / RGW current (matches Ceph release) · Linux kernel 5.15+ (5.10 minimum) · Ubuntu 24.04 LTS (Ceph host baseline) · Debian 12 (Bookworm) (Ceph host baseline) · Rocky Linux / RHEL / AlmaLinux 9.x (Ceph host baseline) · Proxmox VE 9.x (cross-course integration) · Kubernetes 1.31+ (cross-course integration) · 2026-08-18

Why this matters in production

rbd-nbd sits between krbd and librbd: a block device the kernel presents, backed by a user-space process using librbd. That combination solves a specific and recurring problem.

How it works

flowchart LR
  A[Application] --> B["/dev/nbd0"]
  B --> C[kernel NBD driver]
  C --> D[rbd-nbd process]
  D --> E[librbd]
  E --> F[Ceph cluster]

The kernel sees an ordinary block device; the actual Ceph work happens in a user-space process. The application gets a block device without the kernel needing an RBD implementation.

When it is right

Situation	Why rbd-nbd
The kernel is too old for a needed image feature	librbd supports everything
A block device is required, not a library	applications, filesystems
Client bugs must not affect the kernel	the process can be killed
Per-image client configuration is needed	librbd settings apply
Testing librbd behaviour with a block device	matches production librbd

The first is the most common: a feature such as object-map that krbd does not support on the available kernel, needed for a workload that requires a block device.

Deploying

rbd-nbd map rbd-vms/vm-disk-1
# /dev/nbd0

rbd-nbd list-mapped
rbd-nbd unmap /dev/nbd0

# with options
rbd-nbd --device /dev/nbd3 --timeout 120 map rbd-vms/vm-disk-1

# persistent, via systemd
systemctl enable --now rbd-nbd@rbd-vms/vm-disk-1

The --timeout option matters: without it, a cluster interruption can leave the NBD device waiting indefinitely and the filesystem above it hung.

Failure modes

Failure	Behaviour
rbd-nbd process killed	the device disappears; I/O errors
Cluster unreachable	I/O blocks until the timeout, then errors
Process OOM-killed	same as killed
Host reboot	the mapping is not persistent unless configured

# guard against OOM
systemctl set-property rbd-nbd@.service OOMPolicy=continue

The process being killed is the failure mode that distinguishes rbd-nbd from krbd: a kernel client cannot be killed, and a user-space one can. Protecting it from the OOM killer is worth doing explicitly.

ps -o pid,oom_score_adj,cmd -C rbd-nbd

Quiz

Knowledge check · 4 questions

Q1. What problem does rbd-nbd primarily solve?

A block device with librbd's full feature support, decoupled from the kernel's release cycle Higher sequential throughput than krbd, by avoiding the kernel block layer on the data path entirely Lower memory use than krbd, since the cache lives in one user-space process per host Shared access to one image from several hosts at once, which krbd cannot provide safely
Q2. Like krbd, the rbd-nbd client cannot be killed by the OOM killer.

True False
Q3. Choose a client when the kernel is the constraint.

An application needs a block device and benefits from the object-map feature. The enterprise distribution's frozen kernel does not support object-map in krbd, and the kernel cannot be changed.
Q4. What happens without `--timeout` when the cluster becomes unreachable?

Passing score: 75%. Answers are checked in this browser.

Production discipline

Reach for rbd-nbd whenever the kernel version is the constraint on image features — it ships with Ceph and is decoupled from the kernel release cycle. Always set --timeout and protect the process from the OOM killer; both are one line and both are routinely omitted.

Cross-course references

Kubernetes: userspace CSI drivers decouple from node kernel versions the same way
Linux: FUSE filesystems trade kernel integration for release independence