Device inventory and OSD creation

Advanced⏱ ~18 minceph

What you'll learn

Read device inventory and availability
Diagnose a device reported as unavailable
Write drive group specifications
Deploy OSDs with DB and WAL separation

Prerequisites

None — start here.

Verified against Ceph Tentacle 20.2.x · Ceph Squid 19.2.x (supported previous) · cephadm matches the verified Ceph release · podman 4.x · csi-rbd and csi-cephfs current · RBD / CephFS / RGW current (matches Ceph release) · Linux kernel 5.15+ (5.10 minimum) · Ubuntu 24.04 LTS (Ceph host baseline) · Debian 12 (Bookworm) (Ceph host baseline) · Rocky Linux / RHEL / AlmaLinux 9.x (Ceph host baseline) · Proxmox VE 9.x (cross-course integration) · Kubernetes 1.31+ (cross-course integration) · 2026-08-18

Why this matters in production

“Why is this disk not available?” is one of the most common cephadm questions, and the answer is nearly always one of four conditions the inventory reports. Drive groups then turn OSD creation from a per-device command into a declaration that applies to hosts you have not built yet.

Reading inventory

ceph orch device ls
ceph orch device ls --wide --refresh

HOST         PATH      TYPE  DEVICE ID          SIZE  AVAILABLE  REJECT REASONS
ceph-osd-01  /dev/sdb  hdd   SEAGATE_ST16000    16T   Yes
ceph-osd-01  /dev/sdc  hdd   SEAGATE_ST16000    16T   No         LVM detected, locked
ceph-osd-01  /dev/nvme0n1 ssd SAMSUNG_MZQL2     3.8T  No         Insufficient space (<10 GB)

The REJECT REASONS column is the answer.

Reason	Cause
`LVM detected`	existing LVM metadata; zap it
`locked`	mounted, or in use by another subsystem
`Has a FileSystem`	an existing filesystem signature
`Insufficient space`	smaller than the minimum
`Has partitions`	partition table present

Clearing a device

ceph orch device zap ceph-osd-01 /dev/sdc --force

This destroys everything on the device. Confirm the device is the one you intend — the path can change across reboots, so match on the device ID where possible.

Drive groups

Rather than creating OSDs device by device:

service_type: osd
service_id: hdd-with-nvme-db
placement:
  label: osd
spec:
  data_devices:
    rotational: 1
  db_devices:
    rotational: 0
    limit: 2
  db_slots: 6

ceph orch apply -i osdspec.yaml
ceph orch apply osd --all-available-devices --dry-run

This says: on every labelled host, make each spinning disk an OSD, and put their BlueStore DB on the NVMe devices, six DB partitions per NVMe.

The declaration applies to hosts added later, so a new OSD host with the right label is provisioned automatically.

Selection criteria

Criterion	Example
`rotational`	`0` for flash, `1` for spinning
`size`	`'16T:'`, `':2T'`, `'1T:4T'`
`model`	matches the device model string
`vendor`	matches the vendor
`paths`	explicit device paths
`limit`	maximum devices to use
`all`	every matching device

Verifying before applying

ceph orch apply -i osdspec.yaml --dry-run

The dry run shows exactly which devices on which hosts would become OSDs. Run it before every drive group change — a specification matching more than intended consumes devices irreversibly.

Quiz

Knowledge check · 4 questions

Q1. A device shows AVAILABLE No with reject reason "LVM detected". What does this mean?

The device has failed and the orchestrator will not build an OSD on it It is smaller than the minimum size the orchestrator will build an OSD on It carries existing LVM metadata and must be zapped before it can be used It already backs an OSD, which is why the orchestrator will not reuse it
Q2. Specifying db_slots makes DB partition sizes deterministic across hosts with different disk counts.

True False
Q3. Deploy OSDs across a heterogeneous fleet.

A cluster has two host types: twelve HDDs plus two NVMe, and twenty-four HDDs plus two NVMe. A single drive group specification will be applied to both. The team has not set db_slots.
Q4. Why should a drive group specification always be dry-run first?

Passing score: 75%. Answers are checked in this browser.

Production discipline

Dry-run every drive group change; the specification is declarative and destructive, and a selector matching more than intended consumes devices irreversibly. Set db_slots explicitly on heterogeneous fleets so DB partition sizes are deterministic rather than a function of each host’s disk count.

Cross-course references

Kubernetes: dry-run before applying manifests serves the identical purpose
Linux: any partitioning automation warrants the same preview-before-commit discipline