Safely remove a storage host

This runbook covers Safely remove a storage host on a Ceph Tentacle cluster.

When to use

• Active production scenario matching the impact (data-loss-risk)
• The prerequisites are met

Prerequisites

Before starting this runbook, verify each prerequisite has been met. If a prerequisite is not met, resolve it first or escalate.

• The host is to be decommissioned
• Replacement capacity is in place (or cluster headroom is sufficient)
• A maintenance window is scheduled

Pre-checks

Collect evidence before taking action. Evidence is the proof the runbook worked.

# The host passes cephadm check-host
# The cluster is HEALTH_OK
# The clusters capacity headroom is at least 30%
# Time and DNS are healthy

Procedure

Step 1

Mark all OSDs on the host out: ceph osd out osd.<id>

Step 2

Wait for backfill: ceph progress

Step 3

Stop the daemons: ceph orch daemon stop osd.<id>

Step 4

Remove the host: ceph orch host rm <hostname>

Step 5

Verify the host is removed

Step 6

Decommission the host physically

Verification

After completing the procedure, verify the result:

# ceph -s reports HEALTH_OK
# The host is removed from the orchestrator and the CRUSH map
# No PGs are degraded
# ceph osd tree shows the remaining OSDs in the right CRUSH buckets

Rollback

If the procedure does not produce the expected verification, roll back:

• If backfill stalls: tune recovery settings
• If a PG is stuck on the hosts OSDs: identify why the OSD is unresponsive
• If headroom is insufficient: add capacity before removing the host

Escalation

If you cannot roll back or the rollback fails, escalate:

• Storage team