Image metadata and the objects behind it

Advanced⏱ ~17 minrbdrados

What you'll learn

Interpret every field of rbd info
Locate an image's metadata objects in RADOS
Use custom image metadata for per-image settings
Recognise the consequences of metadata object loss

Prerequisites

None — start here.

Verified against Ceph Tentacle 20.2.x · Ceph Squid 19.2.x (supported previous) · cephadm matches the verified Ceph release · podman 4.x · csi-rbd and csi-cephfs current · RBD / CephFS / RGW current (matches Ceph release) · Linux kernel 5.15+ (5.10 minimum) · Ubuntu 24.04 LTS (Ceph host baseline) · Debian 12 (Bookworm) (Ceph host baseline) · Rocky Linux / RHEL / AlmaLinux 9.x (Ceph host baseline) · Proxmox VE 9.x (cross-course integration) · Kubernetes 1.31+ (cross-course integration) · 2026-08-18

Why this matters in production

Everything rbd info reports is stored in RADOS objects, and knowing which objects those are is what makes a damaged image diagnosable rather than mysterious. The custom metadata facility is also the supported way to set per-image client behaviour, which is under-used.

Reading rbd info

rbd info rbd-vms/vm-disk-01

Field	Meaning
`size`	logical size and object count
`order`	object size as a power of two — 22 means 4 MiB
`id`	internal image identifier
`block_name_prefix`	the prefix of this image’s data objects
`format`	2 on any modern cluster
`features`	enabled feature bits
`op_features`	operation features such as clone-v2
`flags`	transient state, e.g. `object map invalid`
`parent` / `overlap`	clone relationship, if any
`data_pool`	present when data is in a separate pool

flags is worth watching: object map invalid means the map is not trusted and operations are using the slow path until it is rebuilt.

The objects behind it

rados -p rbd-vms ls | grep -v rbd_data
# rbd_directory
# rbd_id.vm-disk-01
# rbd_header.1f2a3b4c5d6e
# rbd_object_map.1f2a3b4c5d6e
# rbd_info

Object	Contents
`rbd_directory`	pool-wide image name-to-id mapping
`rbd_id.<name>`	maps this image’s name to its id
`rbd_header.<id>`	size, features, snapshots, parent — the image’s core metadata
`rbd_object_map.<id>`	the object existence bitmap
`rbd_data.<id>.*`	the actual data

rados -p rbd-vms listomapvals rbd_header.1f2a3b4c5d6e | head -20

The header stores its contents in omap, which is why RBD metadata pools must be replicated rather than erasure coded.

Custom image metadata

rbd image-meta set  rbd-vms/vm-disk-01 conf_rbd_cache true
rbd image-meta set  rbd-vms/vm-disk-01 conf_rbd_cache_size 134217728
rbd image-meta list rbd-vms/vm-disk-01
rbd image-meta get  rbd-vms/vm-disk-01 conf_rbd_cache

Keys prefixed conf_ override librbd configuration for that image only — the supported way to give one image different cache or queue settings without changing anything cluster-wide. Arbitrary keys can also be stored for your own tooling:

rbd image-meta set rbd-vms/vm-disk-01 owner-team platform
rbd image-meta set rbd-vms/vm-disk-01 template ubuntu-2404@v1

That second example is a practical answer to the clone-lineage tracking problem — the record travels with the image.

Quiz

Knowledge check · 4 questions

Q1. Which RADOS object holds an RBD image's size, feature set, and snapshot list?

rbd_directory rbd_id.<name> rbd_header.<id> rbd_data.<id>.0000000000000000
Q2. Renaming an RBD image changes its block_name_prefix.

True False
Q3. Give one image different client-side settings.

One RBD image backs a latency-sensitive database while the rest of the pool serves general VMs. The database benefits from a larger librbd cache, but changing the cluster-wide setting would affect every image and increase memory use across all hypervisors.
Q4. Why must an RBD metadata pool be replicated rather than erasure coded?

Passing score: 75%. Answers are checked in this browser.

Production discipline

Use rbd image-meta for per-image client settings and for tracking information such as template lineage or owning team — the record travels with the image rather than living in host configuration that drifts. Include the metadata pool in durability planning for EC-backed images; it holds few bytes and all of the meaning.

Cross-course references

Kubernetes: PVC annotations serve the same role of attaching metadata to the volume
Linux: filesystem superblock loss has the identical data-present-but-unusable character