Removing the host

Intermediate⏱ ~16 mincephcephadm

What you'll learn

Remove a host from the cluster
Handle non-OSD daemons on it
Verify the removal
Clean up the host itself

Prerequisites

None — start here.

Verified against Ceph Tentacle 20.2.x · Ceph Squid 19.2.x (supported previous) · cephadm matches the verified Ceph release · podman 4.x · csi-rbd and csi-cephfs current · RBD / CephFS / RGW current (matches Ceph release) · Linux kernel 5.15+ (5.10 minimum) · Ubuntu 24.04 LTS (Ceph host baseline) · Debian 12 (Bookworm) (Ceph host baseline) · Rocky Linux / RHEL / AlmaLinux 9.x (Ceph host baseline) · Proxmox VE 9.x (cross-course integration) · Kubernetes 1.31+ (cross-course integration) · 2026-08-18

Why this matters in production

A host may run daemons other than OSDs, and removing it without relocating them removes those services.

Checking what runs on it

ceph orch ps --hostname ceph-07

NAME            HOST     STATUS    VERSION
mon.ceph-07     ceph-07  running   19.2.0
mgr.ceph-07     ceph-07  running   19.2.0
crash.ceph-07   ceph-07  running   19.2.0

Daemon present	Action before removing the host
MON	relocate or reduce the count deliberately
MGR	relocate; ensure at least one remains elsewhere
MDS	relocate; ensure standby count is maintained
RGW	relocate; check the service placement
`crash`, `node-exporter`	removed with the host; no action

# relocate a monitor by changing the placement
ceph orch apply mon --placement="ceph-01,ceph-02,ceph-03"
ceph quorum_status

Verifying quorum after a monitor relocation and before removing the host is essential.

Removing the host

ceph orch host rm ceph-07
ceph orch host ls

# if daemons remain, the removal is refused
ceph orch host rm ceph-07 --force

The refusal is protective; forcing it removes the host with daemons still defined on it, which leaves the cluster expecting services that are gone.

Cleaning the CRUSH map

ceph osd crush tree | grep -A2 ceph-07
ceph osd crush remove ceph-07

An empty host bucket remains in the CRUSH map after the OSDs are removed. Leaving it is harmless; removing it keeps the tree accurate.

# and an empty rack, if the node was the last in it
ceph osd crush tree | grep -A3 rack4
ceph osd crush remove rack4

Cleaning the host itself

# on the host, after removal from the cluster
# the cluster fsid: ceph fsid on a remaining host, or the directory
# name under /var/lib/ceph on this one
FSID=3f8a1e2c-9b4d-11ee-b9d1-0242ac120002

cephadm rm-cluster --fsid "$FSID" --force

# verify nothing remains
ls /var/lib/ceph/
systemctl list-units 'ceph*'
podman ps -a | grep ceph

For a host being repurposed this matters; for one being scrapped it does not, except where the devices might be reused elsewhere and would carry Ceph metadata.

# clear devices for reuse
ceph orch device zap ceph-07 /dev/sdb --force
# or, on the host directly
wipefs -a /dev/sdb

Quiz

Knowledge check · 4 questions

Q1. Why does `ceph orch host rm` refuse while daemons are defined on the host?

The check exists because the host may still be named by a service spec, which would recreate its daemons elsewhere Forcing it removes the host while the cluster still expects those services — a monitor removed this way reduces quorum without a clean monmap update The running daemons hold open cephx sessions, and removing the host while those are held leaves the tickets valid until they expire on their own schedule The orchestrator cannot stop containers on a host it has dropped from its inventory, so they would run on unmanaged
Q2. An empty host bucket left in the CRUSH map affects placement.

True False
Q3. Remove a decommissioned host.

A node has been drained and its OSDs removed. It also runs a monitor and a manager. It is the last host in its rack.
Q4. Which daemons require relocation before a host is removed, and which do not?

Passing score: 75%. Answers are checked in this browser.

Production discipline

Relocate monitors and managers and verify their services before removing a host — ceph orch host rm refuses for a reason and forcing it leaves the cluster expecting services that are gone. Remove the empty host and rack buckets from CRUSH afterwards to keep the topology an accurate description.

Cross-course references

Kubernetes: draining a node with control plane components requires relocating them first
Linux: removing a cluster member requires its services to move first