Common Proxmox incident scenarios

Advanced⏱ ~18 min

What you'll learn

Recognise common incident signatures
Apply targeted fixes
Build a troubleshooting cheat sheet for the team
Know when to escalate

Prerequisites

Proxmox troubleshooting methodology

Verified against Proxmox VE 9.2.4 · Proxmox Backup Server 4.2.5 · Ceph Squid / Tentacle · Debian 13 (Trixie) · Linux kernel 7.0 (PVE 9.2 default) · 2026-08-07

Why this matters in production

Most incidents fall into recognisable patterns. Knowing them lets you triage in minutes, not hours.

VM will not start

Symptom: qm start <vmid> returns TASK ERROR.

Cause	How to verify	Fix
VM is locked	`qm status <vmid>` shows lock	Find and clear the lock (`/run/qemu-server/<vmid>.lock`)
Bridge does not exist	Error message references bridge	Restore the bridge or change VM’s bridge
Storage missing	Error references storage	Restore storage or migrate VM
Disk full	`df` on host storage	Free space, then start
QEMU process wedged	`ps` shows stuck QEMU	Kill and restart

tail -n 200 /var/log/pve/qemu/&lt;vmid&gt;.log

Cluster loses quorum

Already covered in the quorum-loss lesson. Recap: diagnose, restore nodes, do not manipulate expected votes.

Corosync problems

Symptoms:

Nodes appear offline intermittently.
pvecm status shows nodes dropping and rejoining.

journalctl -u corosync --since '5 minutes ago' && corosync-cmapctl | grep -E 'members|rings'

Cause	Fix
Network issues (switch, NIC, VLAN)	Coordinate with network team
Firewall blocking UDP 5404-5405	Open the ports
NTP drift	Restore NTP, restart corosync

Storage full

Symptoms: writes fail, services crash, PBS datastore full alert.

Read-only / Safe— Find the full filesystem.

df -h | head && zfs list -o space | head && ceph df 2>/dev/null

Fixes:

Delete unused snapshots.
Expand storage (add OSDs, expand LUNs).
Move data to a different pool.

Ceph HEALTH_WARN

Already covered in the Ceph lessons. Recap:

nearfull → add capacity.
Degraded → restore OSDs.
Slow ops → find the bottleneck.

Slow backup

Cause	Fix
PBS datastore full	Expand; check retention
Network bottleneck between PVE and PBS	Use dedicated network
Source VM doing too much I/O	Schedule differently; freeze the guest
Compression contention	Reduce compression level; offload to a faster CPU

VM has no network

Already covered in the networking lesson. Recap:

Bridge without uplink.
VLAN mismatch.
MTU mismatch.
VM firewall blocking.

A break/fix exercise

Break/Fixintermediate10 minvm

VM start fails with "bridge does not exist"

Symptoms

qm start 100 fails: 'TASK ERROR: bridge 'vmbr1' does not exist'
VM 100 was running before; the host was rebooted recently
Other VMs on the cluster are unaffected

Available evidence

VM 100 config: net0: virtio,bridge=vmbr1
ip -br link show | grep vmbr shows only vmbr0
/etc/network/interfaces shows vmbr0 but not vmbr1

Show diagnosis & remediation

Root cause

The bridge vmbr1 was either renamed or removed during a recent network reconfiguration. The VM's config still references it.

Safe remediation

Either: (1) re-create vmbr1 in /etc/network/interfaces and ifreload -a; (2) edit the VM's network config to use an existing bridge (qm set 100 --net0 virtio,bridge=vmbr0). Choose (1) if vmbr1 was intended, (2) if the network design changed.

Verification

qm start 100 succeeds. The VM has network connectivity. Network test passes.

Prevention

Avoid deleting bridges while VMs reference them. If decommissioning SDN zones, migrate affected VMs first.

Production considerations

Common mistakes

“Let me reboot and see if it fixes it.”
Skipping the recent-changes check.
Working on the wrong subsystem.

Key takeaways

Most incidents fit a recognisable pattern.
Cheat sheets reduce MTTR.
Escalate when stuck.

Knowledge check

Knowledge check · 3 questions

Q1. A VM that was running yesterday now fails to start with a bridge error. What is the first thing to check?

Storage capacity on the node it runs on The guest operating system PBS backups for the guest Recent network configuration changes
Q2. If you can't find the cause within 30 minutes, keep investigating alone.

True False
Q3. Where is the per-VM QEMU log?

Passing score: 75%. Answers are checked in this browser.