Migrating from hyper-converged to dedicated

Why this matters in production

The architecture that suited six nodes may not suit sixteen, and the migration is possible without downtime if it is planned as a gradual transition rather than a cutover.

The signals

Any two of these together is usually enough.

Planning the transition

Phase 1: add dedicated Ceph nodes to the existing cluster
Phase 2: migrate data onto them by draining the hyper-converged OSDs
Phase 3: remove the OSDs from the Proxmox nodes
Phase 4: the Proxmox nodes become compute-only

No cutover, no second cluster, no data migration between clusters — the existing Ceph cluster simply changes which hosts hold its OSDs.

# capacity requirement before starting
ceph df
# the new nodes must hold everything the old OSDs hold

Executing it

# 1. add the dedicated hosts
ceph orch host add ceph-01 10.0.1.21
ceph orch host label add ceph-01 osd
ceph orch daemon add osd ceph-01:/dev/sdb
# ... all devices, all new hosts

# 2. wait for the rebalance to complete
watch -n 60 'ceph -s | grep -E "misplaced|health"'

# 3. drain the hyper-converged OSDs, one host at a time
ceph orch osd rm $(ceph osd ls-tree pve-01 | tr '\n' ' ')
watch -n 60 'ceph orch osd rm status'

# 4. verify before moving to the next host
ceph -s
ceph osd tree | grep -A5 pve-01

One host at a time, waiting for health between each, means the cluster is never carrying more than one drain’s worth of movement.

# 5. move the monitors and managers to the dedicated hosts
ceph orch apply mon --placement="ceph-01,ceph-02,ceph-03"
ceph orch apply mgr --placement="ceph-01,ceph-02"

Verifying the result

ceph orch ps --daemon-type osd | awk '{print $2}' | sort -u
ceph orch ps --daemon-type mon
ceph osd tree

Expected end state:
  no OSDs on Proxmox nodes
  monitors and managers on the dedicated hosts
  Proxmox nodes reachable to Ceph over the storage network only
  VMs unaffected throughout

Quiz

Production discipline

Treat the migration as a sequence of ordinary rebalances within one cluster rather than a cutover — add dedicated hosts, drain the co-located OSDs one host at a time, then move the monitors. Waiting for health between each drain keeps every step bounded and recoverable.

Cross-course references

• Kubernetes: migrating from co-located to dedicated storage nodes uses the same drain pattern
• Linux: gradual member replacement is how any redundant system is re-hosted