Moving a monitor to a different address

Why this matters in production

A monitor is located by the address in the monmap, not by its name, so a host renumbering that nobody propagated leaves a monitor the cluster cannot find and clients that fail only when they next start.

Addresses, not names

ceph mon dump | grep -E '^[0-9]+:'

0: [v2:10.0.1.11:3300/0,v1:10.0.1.11:6789/0] mon.ceph-a
1: [v2:10.0.1.12:3300/0,v1:10.0.1.12:6789/0] mon.ceph-b

The safe path: remove, renumber, re-add

With quorum intact this needs no downtime and no map surgery.

ceph mon ok-to-stop ceph-b
ceph orch daemon rm mon.ceph-b --force
ceph mon dump | grep -E '^[0-9]+:'        # confirm it is gone

# renumber the host, then let cephadm rebuild the monitor there
ceph orch apply mon --placement="ceph-a,ceph-b,ceph-c"
ceph orch ps --daemon-type mon --refresh

# a single address correction while the monitor is otherwise fine
ceph mon set-addrs ceph-b '[v2:10.0.2.12:3300,v1:10.0.2.12:6789]'
ceph mon dump | grep ceph-b

Changing every address at once

When the whole cluster moves subnet there is no live quorum to work through, so the map is edited offline and injected into each monitor.

# take the map while the cluster is still up
ceph mon getmap -o /tmp/monmap
monmaptool --print /tmp/monmap

for m in ceph-a ceph-b ceph-c; do
  monmaptool --rm "$m" /tmp/monmap
done
monmaptool --addv ceph-a '[v2:10.0.2.11:3300,v1:10.0.2.11:6789]' /tmp/monmap
monmaptool --addv ceph-b '[v2:10.0.2.12:3300,v1:10.0.2.12:6789]' /tmp/monmap
monmaptool --addv ceph-c '[v2:10.0.2.13:3300,v1:10.0.2.13:6789]' /tmp/monmap
monmaptool --print /tmp/monmap

# stop every monitor first, then on each host with its own id
sudo -u ceph ceph-mon -i ceph-a --inject-monmap /tmp/monmap
sudo -u ceph ceph-mon -i ceph-a --public-addr 10.0.2.11

Injecting into a subset produces two disjoint views of the monitor set
and no quorum at all. Stop all of them, inject into all of them.

Everything else that has to be told

Quiz

Production discipline

Prefer removing and re-adding a monitor over editing its address, and reserve monmap injection for the case where the whole set moves at once — then stop every monitor before injecting into any of them. Push client configuration in the same window as the renumbering; the client breakage otherwise arrives hours later and looks like something else.

Cross-course references

• Kubernetes: a Service endpoint change propagates to running pods and strands only newly scheduled ones
• Linux: bootstrap configuration that is never validated drifts silently until the next restart