Kernel RBD and librbd compared

Intermediate⏱ ~17 minrbdceph

What you'll learn

Compare the two clients on capability and performance
Choose the right one for a use case
Recognise the feature compatibility constraints
Diagnose issues specific to each

Prerequisites

None — start here.

Verified against Ceph Tentacle 20.2.x · Ceph Squid 19.2.x (supported previous) · cephadm matches the verified Ceph release · podman 4.x · csi-rbd and csi-cephfs current · RBD / CephFS / RGW current (matches Ceph release) · Linux kernel 5.15+ (5.10 minimum) · Ubuntu 24.04 LTS (Ceph host baseline) · Debian 12 (Bookworm) (Ceph host baseline) · Rocky Linux / RHEL / AlmaLinux 9.x (Ceph host baseline) · Proxmox VE 9.x (cross-course integration) · Kubernetes 1.31+ (cross-course integration) · 2026-08-18

Why this matters in production

The choice affects which RBD features can be used, how the client behaves under cluster problems, and where the performance ceiling sits.

The comparison

	krbd	librbd
Location	kernel module	user-space library
Used by	bare-metal mounts, Kubernetes CSI	QEMU/KVM, applications
Client cache	page cache	librbd cache
Feature support	lags the release	current
Per-image overhead	low	moderate
Sequential throughput	typically higher	slightly lower
Crash behaviour	a bug can panic the host	a bug affects one process
Configuration	kernel parameters, mount options	rbd config, ceph.conf
Upgrade cadence	tied to the kernel	tied to the Ceph packages

Feature compatibility

rbd info rbd-vms/vm-disk-1 | grep features
rbd feature disable rbd-vms/vm-disk-1 object-map fast-diff deep-flatten

krbd supports a subset of image features, and the set depends on the kernel version. Mapping an image with unsupported features fails:

rbd: map failed: (6) No such device or address
dmesg: rbd: image uses unsupported features: 0x38

# create images for krbd with a compatible feature set
rbd create --size 100G --image-feature layering rbd-vms/krbd-image

The features most often needing to be disabled are object-map, fast-diff, deep-flatten, and exclusive-lock on older kernels.

Performance characteristics

# krbd
rbd map rbd-vms/bench
fio --name=krbd --filename=/dev/rbd0 --direct=1 --rw=randread \
    --bs=4k --iodepth=32 --runtime=60 --time_based

# librbd
fio --name=librbd --ioengine=rbd --pool=rbd-vms --rbdname=bench \
    --rw=randread --bs=4k --iodepth=32 --runtime=60 --time_based

krbd generally achieves higher sequential throughput because it avoids the user-space copy and integrates with the kernel’s readahead and page cache. librbd is competitive on random workloads and offers features krbd lacks.

Diagnosing each

# krbd
dmesg -T | grep -i rbd
cat /sys/bus/rbd/devices/0/client_addr
rbd showmapped

# librbd
ceph --admin-daemon /var/run/ceph/ceph-client.*.asok perf dump
ceph --admin-daemon /var/run/ceph/ceph-client.*.asok objecter_requests

The librbd admin socket is a significant diagnostic advantage: it exposes in-flight requests and detailed counters that krbd does not offer.

Quiz

Knowledge check · 4 questions

Q1. Why do hypervisors typically use librbd despite krbd's throughput advantage?

librbd achieves higher random IOPS, because it bypasses the kernel block layer entirely krbd cannot create or roll back snapshots, so any image needing them must use librbd librbd uses less memory per image, since it shares one cache across every image a host opens A librbd fault kills one process while a krbd fault runs in kernel context and can hang or crash the host
Q2. An image created with default features will always map successfully with krbd.

True False
Q3. Choose a client for a new deployment.

A team is deploying bare-metal database servers that will mount RBD images directly. They ask whether to use krbd or librbd via rbd-nbd.
Q4. What diagnostic capability does librbd offer that krbd does not?

Passing score: 75%. Answers are checked in this browser.

Production discipline

Create images destined for krbd with a feature set verified against the oldest kernel in the fleet; the mapping failure names a bitmask rather than a feature and is confusing to diagnose. Prefer librbd where a client fault must not take the host down — the blast radius difference is the main reason hypervisors use it.

Cross-course references

Kubernetes: CSI drivers face the same kernel-versus-userspace blast radius trade
Linux: in-kernel versus FUSE filesystems present the identical choice