Welcome to RunBook Academy · Proxmox VE

This short lesson exists to demonstrate every component of the design system. Skim it once; then dive into the curriculum proper.

What this course is

A practical, production-oriented curriculum for Proxmox VE administrators. Every lesson follows the same pattern: mental model → visual architecture → GUI + CLI walkthroughs → hands-on lab → break/fix exercise → references.

Verified against

Severity model

Every command example is tagged with a severity badge. This is not decoration — it is a production discipline tool.

pvecm status

systemctl edit pveproxy

systemctl restart pveproxy

pveceph osd destroy 5

zpool destroy tank

pvecm delnode pve-04

Lesson anatomy

Major lessons follow this structure:

What you will learn
Why this matters in production
Mental model
Architecture / visual explanation
Concepts
How Proxmox implements it
GUI walkthrough
CLI walkthrough
Important configuration files
Hands-on lab
Verify your work
Break/fix exercise
Troubleshooting
Production considerations
Common mistakes
Security considerations
Performance considerations
Key takeaways
Knowledge check
Further reading

A lab callout

Lab

Mode A · Physical hardwareMode B · Nested virtualisationMode C · Simulation / guided exercise

This is the lab pattern. Every lab declares its supported modes:

Mode A — physical hardware (most realistic)
Mode B — nested virtualisation (works on capable workstations)
Mode C — simulation/guided exercise (always available)

A break/fix scenario

Break/Fixintermediate15 minvm

A VM fails to start after a network change

Symptoms

qm start 104 returns TASK ERROR: bridge 'vmbr1' does not exist
VM config references a bridge that was removed during SDN reconfiguration
Migration also fails for the same VM

Available evidence

/etc/pve/qemu-server/104.conf contains: net0: virtio=...,bridge=vmbr1
pvesh get /nodes/localhost/qemu/104/config returns the same bridge reference
ip link show no longer lists vmbr1

Show diagnosis & remediation

Root cause

The VM's network interface references a bridge that no longer exists after an SDN cleanup. Proxmox validates the bridge name at start time and refuses to launch the VM.

Safe remediation

Two valid paths: (1) edit the VM config (qm set 104 --net0 virtio,bridge=vmbr0) to point at a current bridge, or (2) re-create the missing bridge if the SDN layer was decommissioned accidentally. The right choice depends on whether the VM's network traffic actually depended on the deleted bridge.

Verification

Run `qm start 104` and confirm the VM starts cleanly. Verify network connectivity from inside the guest.

Prevention

Avoid deleting bridges while VMs reference them. If decommissioning SDN zones, first move or migrate affected VMs, then verify `grep -rl 'bridge=vmbr1' /etc/pve/` returns nothing before removing the bridge.

Callout types

Interactive simulators (click any to load)

These islands prove the visual/teaching principle: difficult concepts become tractable when you can manipulate them.

Quorum simulator

Interactive · Quorum Simulator

Toggle node failures and network link losses to see whether the cluster keeps quorum. The goal of this exercise is to build intuition for the difference between alive nodes and quorate partition.

Topology: 3 nodes × 1 vote each. Expected = 3, quorum = 2.

Click a node circle to toggle alive/down. To simulate network partition, click a link below.

Cluster state

QUORATE

Expected: 3
Alive: 3
Needed: 2

Safe. Cluster-wide operations (VM start/stop, migration, HA actions) can proceed. Self-fencing is not triggered.

What is happening mathematically?

Corosync uses a majority-vote model. With 3 expected votes, quorum requires ⌊3/2⌋ + 1 = 2 votes. With a QDevice, the cluster tolerates one PVE-node failure even in a 2-node setup because the QDevice breaks the symmetric partition.

High Availability behaviour

Interactive · HA Behaviour Simulator

Click a node to mark it as failed. Watch how the cluster resource manager (CRM) steals the HA-managed VMs and restarts them on a surviving node. Note the watchdog state — if quorum is lost, the watchdog triggers a self-reset on the failed node.

Cluster nodes

HA-managed VMs (current placement)

VMID	Name	Preferred	Current node	State
101	web-01	pve-01	pve-01	started
102	db-01	pve-02	pve-02	started
103	app-01	pve-03	pve-03	started
104	util-01	pve-01	—	stopped

Cluster Resource Manager

CRM master: pve-01
Watchdog: armed
Quorate: yes

All nodes healthy. HA is idle; the CRM watchdog is armed but not triggering.

VM packet flow

Interactive · VM → Wire Packet Flow

Watch a single packet traverse each layer that Proxmox adds between a guest application and the physical network. The end-to-end latency is the sum of every hop plus any protocol overhead.

VM guest

Application socket

+0ms

VirtIO net

Paravirt frontend

+0.05ms

tap interface

Linux tap device

+0.05ms

Linux bridge

vmbr0

+0.1ms

VLAN tag

802.1Q VID 100

+0.02ms

LACP bond

bond0

+0.1ms

NIC driver

ens4 → ixgbe

+0.2ms

Physical switch

Top-of-rack

+0.3ms

End-to-end latency

0.82 ms

This is the steady-state latency for a guest on a healthy Proxmox node. Most "slow VM" reports trace back to one of the eight hops above, often storage rather than network.

Storage write path

Interactive · VM Storage Write Path

A single write() from inside a guest passes through several layers before bytes hit a physical disk. The exact latency and durability depends on the storage backend, sync semantics, and any caching devices.

Backend: sync semantics SLOG device host cache (writeback)

Application

sys_write() / fsync()

Guest filesystem

ext4 (writeback)

VirtIO SCSI

Paravirt disk frontend

QEMU

host-backed or thin

ZFS

mirror vdev (2 disks, ZIL, optional SLOG)

Physical device

NVMe or SATA SSD

Write semantics

Write is acknowledged only after durability is guaranteed

Production pitfall

ZFS always writes transactionally. Adding an SLOG speeds sync writes but does not make async writes faster. Never put ZFS on a hardware RAID — ZFS must see the disks directly to provide correct checksums.

Backup flow

Interactive · PBS Backup & Restore Flow

Each toggle changes the pipeline. Notice that all the integrity layers matter: encryption protects confidentiality, verification detects silent corruption, and off-site sync protects against site loss. A green backup job is not evidence that recovery works — only a tested restore is.

Encryption Compression Scheduled verification Off-site sync Tape export

Source VM

QEMU snapshot pauses writes

stage 1

Proxmox VE node

vzdump reads via QEMU stream API

stage 2

Backup stream

zstd compressed

stage 3

Encryption layer

AES-256-GCM (chunk-level)

stage 4

PBS datastore

chunk store, variable-size dedup

stage 5

Verification task

weekly scheduled verify

stage 6

Remote sync

pushed to off-site PBS

stage 7

Restore target

into isolated lab network

stage 8

Resilience posture

Defence in depth

3 of 3 core integrity layers enabled.

Why encryption + verification + off-site?

Encryption stops a compromised PBS host from reading your data. Verification catches silent bit-rot in the chunk store. Off-site sync survives the loss of the primary site. None of the three is sufficient alone; together they cover confidentiality, integrity, and availability.

Ceph topology

Interactive · Ceph OSD failure & recovery

With a 3-replica pool and 2 OSDs per host, Ceph can tolerate any 2 OSD losses. Toggle OSDs or whole hosts to watch the cluster transition through HEALTH_OK → HEALTH_WARN and the recovery/rebalancing kick in.

Click a host box to fail all OSDs on that host. Click an OSD circle to fail a single OSD. Click Reset to restore.

Ceph status

HEALTH_OK

All OSDs up; cluster idle.

Capacity planner

Interactive · Capacity Planner

Sketch a workload inventory and see the cluster sizing implications, including N+1 HA headroom and a configurable growth multiplier.

Workload inventory

Name	vCPU	RAM (GB)	IOPS	Storage (GB)
Workload name	vCPUs	RAM GB	IOPS	Storage GB
Workload name	vCPUs	RAM GB	IOPS	Storage GB
Workload name	vCPUs	RAM GB	IOPS	Storage GB
Workload name	vCPUs	RAM GB	IOPS	Storage GB
Workload name	vCPUs	RAM GB	IOPS	Storage GB
Workload name	vCPUs	RAM GB	IOPS	Storage GB

Cluster size Growth × N+1 (HA spare)

Sizing recommendations

Cluster nodes: 4
vCPU per node: ~13
RAM per node: ~40 GB
IOPS per node: ~5694
Storage per node: ~383 GB

Includes 40% headroom for growth and 1 spare node for HA failover capacity.

Sanity check: 5694 IOPS/node is the ceiling your storage must sustain during a node failure. HDDs are typically unsuitable for this workload mix; SSDs are mandatory.

Decision tree (storage choice)

Interactive · Decision tree

Answer each question to arrive at a contextual recommendation. There is rarely one "right" answer in infrastructure; the goal is to make the trade-off visible.

Is your environment single-site and small (≤ 20 VMs)?

Knowledge check

Course progress

Your progress is stored in this browser only (localStorage). It is not transmitted to any server. The progress bar in the right sidebar tracks completed lessons and quiz scores. Reset it from the toolkit page if needed.

Where to go next

Open the Curriculum page from the header to see the full lesson map, or jump straight to Part I · Foundations to start with the fundamentals.