What HA requires: the storage, network and fencing matrix

ha-manager add vm:100 succeeds on any guest, on any storage, on any cluster with quorum. The interface shows a green HA tab. ha-manager status reports the service as started. Every dashboard agrees the VM is protected.

None of that is a statement about whether HA can recover it. PVE does not validate the prerequisites at enable time, and the first thing that tests them is a node failure.

This lesson is the check that should happen instead.

The requirements, from the documentation

The official list is short:

at least three cluster nodes (to get reliable quorum), shared storage for VMs and containers, hardware redundancy (everywhere), use reliable “server” components.

Each of those is load-bearing in a different way, and the third and fourth are the ones people read as advice rather than as requirements.

Storage: the matrix

The question a backend must answer is: can a surviving node open this guest’s disks, right now, without the failed node’s cooperation?

set -euo pipefail

pvesm status --output-format json-pretty | head -30

# The declarations, including the shared flag on each entry.
grep -E '^\s*(dir|nfs|cifs|lvm|lvmthin|zfspool|rbd|cephfs|iscsi|pbs)|shared' \
/etc/pve/storage.cfg

What actually happens to a guest on local storage

Not “HA does not work”. Something specific, and knowing which is the difference between a five-minute diagnosis and an hour.

# ha-manager status

quorum OK
master pve-02 (active, Tue Aug 11 03:42:17 2026)
lrm pve-01 (old timestamp - dead?, Tue Aug 11 03:38:02 2026)
lrm pve-02 (active, Tue Aug 11 03:42:19 2026)
lrm pve-03 (active, Tue Aug 11 03:42:20 2026)
service vm:100 (pve-01, error)
service vm:101 (pve-02, started)

The sequence, in order:

1. pve-01 stops sending heartbeats. The CRM notices.
2. The failed node self-fences via its watchdog, or the CRM waits out the fencing timeout. This takes on the order of a minute; it is not instant and it is not meant to be.
3. The CRM moves vm:100 to recovery and looks for a node that can run it.
4. Every surviving node reports that it cannot access the guest’s storage.
5. The service goes to error state and stays there. Nothing further happens automatically.

The guest is down, and it stays down until an operator intervenes. That is the correct behaviour - the alternative is starting a VM with no disks - but it is not what the person who enabled HA was expecting to buy.

Recovering from error state, once you have made the storage available or decided to restore:

set -euo pipefail
VMID=100

ha-manager set "vm:$VMID" --state disabled
# ... resolve the storage or restore the guest ...
ha-manager set "vm:$VMID" --state started

ha-manager status | grep "vm:$VMID"

Fencing: the prerequisite that is not optional

Fencing is the part that makes HA safe rather than merely automatic, and it is the part that gets treated as a detail.

The principle is not specific to Proxmox, and this course does not restate it

• the Linux course develops it properly in why fencing exists and STONITH and data integrity, with the watchdog mechanism in SBD and watchdog fencing. Read those; the reasoning transfers directly.

The one-sentence version, because it must be said here too: HA without working fencing is not high availability, it is a mechanism for putting two writers on one disk. If the CRM cannot be certain the old node has stopped, starting the guest elsewhere means two live instances writing to the same image, and a filesystem subjected to that does not survive it.

Proxmox implements self-fencing through the watchdog. The prerequisites:

The other prerequisites

Three nodes, or two plus a QDevice. Quorum needs a majority. A two-node cluster has no majority after one failure, so the survivor cannot safely act; the QDevice supplies the third vote. Covered in quorum and QDevice.

Capacity for one node’s workload on the survivors. HA cannot start what does not fit. A cluster at 90% utilisation has HA configured and not available.

The guest’s network must exist on every candidate node. A bridge named vmbr1 on pve-01 and absent on pve-03 means recovery onto pve-03 produces a running guest with no network - which is often worse than a stopped one, because monitoring sees it as up.

No node-pinning devices. PCI passthrough, a mapped USB device, a dev[n] entry on a container: all of these pin the guest to hardware. HA will start the guest without the device unless a node-affinity rule prevents it.

A readiness check that can fail

Run this before enabling HA on anything, and after any storage or network change. It is deliberately written so that each check either passes or prints a specific failure.

set -euo pipefail
VMID=100
NODES='pve-01 pve-02 pve-03'

echo '--- quorum ---'
pvecm status | grep -E 'Quorate|Expected votes|Total votes'

echo '--- storage backing this guest ---'
# Storage IDs referenced by any disk line in the guest config.
STORES=$(qm config "$VMID" \
| grep -oE '^(scsi|virtio|sata|ide|efidisk|tpmstate)[0-9]+: [A-Za-z0-9_.-]+:' \
| awk '{print $2}' | tr -d ':' | sort -u)
for STORE in $STORES; do
# type and shared, straight from the storage definition
printf '  %-16s ' "$STORE"
pvesh get "/storage/$STORE" --output-format yaml \
  | grep -E '^(type|shared):' | tr '\n' ' '
printf '\n'
done

echo '--- bridges present on every node ---'
for BR in $(qm config "$VMID" | grep -oE 'bridge=[a-z0-9]+' | cut -d= -f2 | sort -u); do
for N in $NODES; do
  ssh -o BatchMode=yes "root@$N" "ip link show $BR >/dev/null 2>&1" \
    && printf '  %s on %s OK\n' "$BR" "$N" \
    || printf '  %s MISSING on %s\n' "$BR" "$N"
done
done

echo '--- node pinning ---'
qm config "$VMID" | grep -E '^(hostpci|usb)[0-9]+:' \
&& echo '  WARNING: passthrough pins this guest to specific hardware' \
|| echo '  no passthrough'

echo '--- fencing ---'
for N in $NODES; do
printf '  %s watchdog: ' "$N"
ssh -o BatchMode=yes "root@$N" 'ls /dev/watchdog 2>/dev/null || echo MISSING'
done

Four failure classes, four specific messages. That is what distinguishes a readiness check from ha-manager status, which tells you HA is configured and nothing about whether it would work.

One nuance in reading the storage output: shared is absent from the definition of backends that are inherently shared - rbd, cephfs, nfs, cifs all are, by construction. An absent shared line on a dir, lvmthin or zfspool entry is the failure; an absent one on rbd is normal. Read the type first, then the flag.

Knowledge check