Skip to main content
RunBook Academy

← All runbooks in Ceph

medium riskservice affecting~90 min

Deploy a new Ceph cluster with cephadm

1 · Prerequisites

Confirm every item is in place before any state change.

  • A pool of 6+ hosts with Ubuntu 24.04 LTS / Debian 12 / Rocky 9
  • Network: bond + LACP, separate VLANs for public and cluster networks
  • DNS: forward and reverse, every hostname
  • Time: chrony or systemd-timesyncd, offset < 100ms
  • Container runtime: podman 4.x

2 · Pre-checks

Read-only diagnostic commands. If any of these don't match expected output, stop and investigate further.

  • · cephadm --version reports the current version
  • · cephadm check-host --skip admin reports the host is ready
  • · podman --version reports 4.x
  • · Time offset is < 100ms

3 · Procedure

Execute each step in order. Verify the expected output of a step before moving to the next.

  1. 1cephadm bootstrap --mon-ip <mgr-host> --cluster-network <subnet> initialises the cluster
  2. 2Capture the dashboard URL and the bootstrap password
  3. 3ssh -o StrictHostKeyChecking=no <mgr-host> to verify SSH key auth
  4. 4Add each host: ceph orch host add <hostname> <ip>
  5. 5Deploy MONs: ceph orch apply mon --placement "3 hosts"
  6. 6Deploy MGRs: ceph orch apply mgr --placement "3 hosts"
  7. 7Deploy OSDs: ceph orch apply osd --all-available-devices
  8. 8Verify: ceph -s reports HEALTH_OK and the expected number of services

4 · Verification

Confirm the procedure actually fixed the problem.

  • ✓ceph -s reports HEALTH_OK
  • ✓ceph osd tree shows the expected number of OSDs up and in
  • ✓ceph orch ls lists the deployed services
  • ✓ceph orch ps shows daemons running on every host

5 · Rollback

If verification fails, undo the procedure in reverse order.

  • ↶If the bootstrap fails: cephadm bootstrap --uninstall and start over
  • ↶If a host cannot be added: verify SSH, network, and time synchronisation
  • ↶If OSDs do not deploy: verify the disks are not in use by the host OS

6 · Escalation

When the runbook isn't enough, contact:

  • · Storage team if multiple hosts fail to deploy
  • · Vendor support for hardware issues
  • · Escalation procedure per the operations manual

Deploy a new Ceph cluster with cephadm

This runbook covers Deploy a new Ceph cluster with cephadm on a Ceph Tentacle cluster.

When to use

  • Active production scenario matching the impact (service-affecting)
  • The prerequisites are met

Prerequisites

Before starting this runbook, verify each prerequisite has been met. If a prerequisite is not met, resolve it first or escalate.

  • A pool of 6+ hosts with Ubuntu 24.04 LTS / Debian 12 / Rocky 9
  • Network: bond + LACP, separate VLANs for public and cluster networks
  • DNS: forward and reverse, every hostname
  • Time: chrony or systemd-timesyncd, offset < 100ms
  • Container runtime: podman 4.x

Pre-checks

Collect evidence before taking action. Evidence is the proof the runbook worked.

# cephadm --version reports the current version
# cephadm check-host --skip admin reports the host is ready
# podman --version reports 4.x
# Time offset is < 100ms

Procedure

Step 1

cephadm bootstrap —mon-ip <mgr-host> —cluster-network <subnet> initialises the cluster

Step 2

Capture the dashboard URL and the bootstrap password

Step 3

ssh -o StrictHostKeyChecking=no <mgr-host> to verify SSH key auth

Step 4

Add each host: ceph orch host add <hostname> <ip>

Step 5

Deploy MONs: ceph orch apply mon —placement “3 hosts”

Step 6

Deploy MGRs: ceph orch apply mgr —placement “3 hosts”

Step 7

Deploy OSDs: ceph orch apply osd —all-available-devices

Step 8

Verify: ceph -s reports HEALTH_OK and the expected number of services

Verification

After completing the procedure, verify the result:

# ceph -s reports HEALTH_OK
# ceph osd tree shows the expected number of OSDs up and in
# ceph orch ls lists the deployed services
# ceph orch ps shows daemons running on every host

Rollback

If the procedure does not produce the expected verification, roll back:

  • If the bootstrap fails: cephadm bootstrap —uninstall and start over
  • If a host cannot be added: verify SSH, network, and time synchronisation
  • If OSDs do not deploy: verify the disks are not in use by the host OS

Escalation

If you cannot roll back or the rollback fails, escalate:

  • Storage team if multiple hosts fail to deploy
  • Vendor support for hardware issues
  • Escalation procedure per the operations manual

References

  1. Ceph documentation