The pveXtoY upgrade checker and the pre-upgrade gate · Proxmox VE

Proxmox ships a checker with every major upgrade. It is named for the transition — pve7to8, pve8to9 — it is installed on the source version before you need it, and it is the first command in Proxmox’s own documented procedure.

It is also the single highest-value command in a major upgrade, because it converts a category of failure that would otherwise appear halfway through apt dist-upgrade on a production node into a list you can read beforehand, on a node that is still working.

What it is

pve8to9 inspects the node and the cluster and reports on everything Proxmox knows can break an 8-to-9 upgrade. It makes no changes by default — it is a report, and every finding is yours to act on.

Read-only / Safethe checker— Read-only. Inspects the node, the cluster and the storage configuration and prints findings. Changes nothing.

pve8to9

pve8to9 --full

--full runs the extended set, including the slower checks. Use it: the time saved by the short form is minutes, and the checks it skips are not the unimportant ones.

Reading the output

Findings come in five classes and the correct response differs for each.

Class	Meaning	What to do
`PASS`	Checked, and fine	Nothing
`INFO`	Informational context	Read it. Some INFO lines carry version numbers you need later
`SKIP`	Not applicable to this node	Confirm the reason is what you expect
`WARN`	Will not stop the upgrade; may change behaviour after it	Investigate every one, then decide
`FAIL`	Will break the upgrade or the node afterwards	Clear it. This is a stop.

Read-only / Safea checker run with work to do— Illustrative, and shortened. Real output is considerably longer and grouped by section. The final summary line is the number to read first.

# pve8to9 --full

= CHECKING VERSION INFORMATION FOR PVE PACKAGES =

Checking for package updates..
PASS: all packages up-to-date

Checking proxmox-ve package version..
PASS: proxmox-ve package has version >= 8.4.0

= CHECKING CLUSTER HEALTH/SETTINGS =

PASS: systemd unit 'pve-cluster.service' is in state 'active'
PASS: Cluster Filesystem is quorate.
WARN: 1 nodes are offline!

= CHECKING HYPER-CONVERGED CEPH STATUS =

SKIP: no hyper-converged ceph setup detected!

= CHECKING CONFIGURED STORAGES =

PASS: storage 'local' enabled and active.
WARN: storage 'backup-nfs' enabled but not active!

= MISC CHECKS =

PASS: Resolved node IP '192.0.2.11' configured and active on single interface.
FAIL: There is only 3.1 GiB free space on '/'. 5 GiB is the minimum,
    10 GiB or more is recommended.
INFO: Found 2 guest(s) with 'machine' set to a pinned version.

= SUMMARY =

TOTAL:    41
PASSED:   36
SKIPPED:   2
WARNINGS:  2
FAILURES:  1

ATTENTION: Please check the output for detailed information!

Illustrative output

Clearing the common findings

Insufficient free space on `/`

The prerequisite is “at least 5 GB free disk space on the root mount point, ideally more than 10 GB”. This is the most frequent FAIL and the most dangerous to ignore, because running out of space partway through dist-upgrade leaves the package database in a state that is tedious to repair on a node that is no longer serving guests.

Read-only / Safefind where the space went— Read-only. The apt cache and old kernels are the two usual consumers on a Proxmox node; the journal is the third.

df -h /
du -xh --max-depth=1 / 2>/dev/null | sort -rh | head -15
du -sh /var/cache/apt/archives /var/log/journal
proxmox-boot-tool kernel list

Configuration changereclaim it— Removes downloaded package files that are no longer available for download and caps the journal. autoclean is the conservative form; note that a full apt clean also discards packages you might roll back to.

apt autoclean
journalctl --vacuum-size=500M

apt autoremove --purge

If old kernels are the consumer, remove one from the boot set with proxmox-boot-tool kernel remove rather than deleting files from /boot, which on a ZFS-root system is not where the booting kernels live.

An offline node

A WARN about offline nodes is a FAIL in disguise for a cluster upgrade. An upgrade performed while a node is down produces a cluster where the absent node comes back on the old version, into a cluster that has moved — and the version-skew tolerance during a major upgrade is bounded and temporary, not indefinite.

Bring it back, or remove it from the cluster deliberately, before starting.

A storage configured but not active

An inactive storage during an upgrade is usually harmless in itself and is a signal worth chasing: either the storage is genuinely gone and the configuration is stale, or it is temporarily unreachable, in which case a guest referencing it will not start after the reboot.

Read-only / Safewhich storages are actually usable, and who uses them— Read-only. pvesm status reports reachability per storage; the config file names the guests that depend on each.

pvesm status
cat /etc/pve/storage.cfg

Disable a stale definition rather than deleting it, so the change is reversible during the window.

Ceph must be upgraded first

The documentation is explicit: “Hyper-converged Ceph: upgrade any Ceph Quincy or Ceph Reef cluster to Ceph 19.2 Squid before you start the Proxmox VE upgrade to 9.0.” Confirm with ceph --version before proceeding.

This ordering is not a preference. Skipping it puts you in a combination Proxmox does not test, on a storage layer holding every running guest.

Old configuration formats and deprecated settings

The checker reports settings that PVE 9 handles differently — legacy sources.list entries rather than deb822 .sources files, deprecated storage options, guest configuration keys that changed meaning. Each of these is a small, specific edit, and each is much cheaper before the upgrade than as a surprise after it.

Where to run it, and when

Three points, and the third is the one people skip.

Before you touch anything. On every node. Findings differ per node — free space, local storage, pinned machine versions — and a cluster is ready when the last node is, not the first.

After each node upgrades, on the nodes still on the old version. The checker is cluster-aware, and running it mid-upgrade surfaces problems introduced by the mixed-version state.

After the whole cluster is done. The documentation asks for this explicitly, and it is the step that turns “the upgrade completed” into “the upgrade is correct”.

Read-only / Safecheck every node from one place— Read-only. Runs the checker on each node in turn over SSH and captures the output, so the pre-upgrade state of the whole cluster is one artefact rather than several terminal sessions.

for NODE in pve01 pve02 pve03; do
echo "=== $NODE ==="
ssh "root@$NODE" 'pve8to9 --full' 2>&1 | tee "/root/pve8to9-$NODE.txt"
done

grep -c '^FAIL' /root/pve8to9-*.txt

Keep those files. They are the pre-upgrade evidence, and comparing the post-upgrade run against them is how you show the window achieved what it set out to.

What the checker does not check

This is the section that matters most, because a clean run creates confidence proportional to how much you think it covered.

Your guests. The checker verifies the platform. It says nothing about whether the applications inside your VMs tolerate the reboot, whether a guest depends on a QEMU machine version that behaves differently under QEMU 11, or whether a container’s userspace works on the new kernel.

Your automation. Terraform providers, Ansible collections, Proxmoxer, custom scripts against the API — all pinned to an API surface that changes across a major version. Nothing checks these, and the symptom arrives at the next automated run rather than during the window.

Your backups. It does not verify that a restore works. The prerequisite is “a valid and tested backup of all VMs and CTs”, and “tested” is doing the work in that sentence. A restore drill before a major upgrade is the actual safety net; the checker is the gate.

Your monitoring and integrations. Metric server exports, notification targets, an external Ceph client, a PBS server on a version that may need its own upgrade first.

Hardware and firmware. A NIC that needs a firmware level for the new kernel, an HBA with a known issue, a BMC that needs updating. The 6.x-to-7.0 kernel move in PVE 9.2 makes this a live concern.

Your ability to get out. There is no rollback for a major upgrade of a running node. The way back is restore from backup, and the checker does not ask whether you could.

Under the hood

What the checker actually inspects, and why that shapes its blind spots.

It is a Perl script shipped in pve-manager on the source version. That placement has a direct consequence: it is updated by ordinary package updates on PVE 8, so a node that is fully current before the upgrade has a newer, better checker than one that has not applied updates in six months. This is part of why the documented prerequisite is “upgraded to the latest version of Proxmox VE 8.4 on all nodes” before starting.

The checks are grouped into sections you can see in the output: package versions, cluster health, hyper-converged Ceph, configured storages, and a misc group covering free space, the resolved node IP, guest configuration and known-incompatible settings. Each is a direct inspection of local state — package versions from dpkg, cluster state from pmxcfs and corosync, storage reachability through the same pvesm path the GUI uses, free space from the filesystem.

Everything it examines is therefore state the node can see. That is the structural reason for the blind spots: guest application behaviour, the API compatibility of your Terraform provider, whether a restore works, and whether your NIC firmware is current are all outside the node’s view, and no amount of improving the script would bring them in.

Two details worth knowing during a window.

It is cluster-aware but node-local in what it can fix. It reads cluster state through pmxcfs, so it reports on quorum and on other nodes being offline, and it checks the node it runs on for everything local. Running it on one node of a five-node cluster gives you one node’s readiness plus a cluster-wide view — which is why the loop over all nodes is the right shape.

The repository transition is the change underneath everything it checks. PVE 9 is built on Debian 13 Trixie, so the upgrade replaces bookworm with trixie throughout the repository configuration, and PVE 9 also moves to the deb822 .sources format. apt modernize-sources converts most legacy entries automatically after the upgrade. A node whose repositories are hand-maintained or carry third-party entries pinned to bookworm is the case the checker flags and cannot fix, and it is the one that most often stalls a dist-upgrade partway through.

What happens if you skip the checker and upgrade anyway?

Usually it works. That is precisely why the habit erodes: most nodes upgrade cleanly without it, and the checker starts to feel like ceremony until the run where it was not.

The failures it would have caught do not arrive as a refused upgrade. They arrive partway through, on a node that has already stopped being a working hypervisor.

Out of space mid-dist-upgrade. Packages are unpacked and not configured, dpkg is in an inconsistent state, and the recovery is to free space and run dpkg --configure -a — on a node whose guests are already down and whose pve* tooling may not be functioning. Doable, and unpleasant, and long.

A repository still pointing at bookworm. apt resolves a half-Trixie, half-Bookworm dependency set and either refuses or produces a mixture. The refusal is the good outcome.

Ceph not upgraded first. Now you are running a PVE 9 node against a Ceph release combination Proxmox does not test, with every guest’s storage underneath it. There is no quick way back.

A node that was offline. It returns on PVE 8 into a cluster that has moved to 9. Version-skew tolerance during an upgrade is a transitional allowance, not a supported steady state, and the symptoms are intermittent rather than obvious.

The compounding problem in all of these is that a major upgrade of a running node has no rollback. You cannot apt your way back to PVE 8; the path back is reinstall and restore. So a failure discovered at 40% through the upgrade is not a decision point — the decision was made when dist-upgrade started, and the only remaining question is how long the recovery takes.

Which is the actual argument for the checker. It is not that it finds things you could not find yourself. It is that it finds them while going back is still free.

Common mistakes

Running it once, on one node. Findings are per node, and a cluster is ready when the last node is.
Treating WARN as noise. Offline nodes and inactive storages are warnings that become failures after the reboot.
Skipping --full. The checks it omits are not the unimportant ones.
Not running it after the upgrade. That run is what turns “completed” into “correct”.
Reading FAILURES: 0 as “safe to proceed”. It means the platform is ready. Guests, automation, backups and firmware are separate questions.
Upgrading before Ceph. Explicitly documented as the wrong order, on the layer holding every guest.
Running it on a node that has not applied PVE 8 updates. The checker itself is out of date.
No restore drill. The checker is the gate; the tested backup is the safety net, and only one of them gets you out.

Key takeaways

pve8to9 --full is the first command in Proxmox’s own upgrade procedure. It changes nothing and gates everything.
Five result classes. FAIL is a stop; WARN is an investigation; SKIP deserves a glance at why.
Run it before, between nodes, and after. Capture the output as evidence.
Clear findings rather than acknowledging them. Free space, offline nodes, inactive storages and Ceph ordering are the recurring four.
Upgrade Ceph to Squid before starting PVE 9. Documented, and on the layer under every guest.
It cannot see your guests, your automation, your backups or your firmware. Check those yourself.
A major upgrade has no rollback. The checker finds problems while turning back is still free.

Knowledge check

Why this matters

A major version upgrade is the change with the worst risk profile a Proxmox operator performs. It touches the kernel, the base OS, the hypervisor stack and often the storage layer at once. It requires a reboot. And it has no rollback: once dist-upgrade has run, there is no supported path back to the previous version — the way out is reinstall and restore.

That last property is what makes a pre-upgrade gate different in kind from a checklist. For most changes, the plan is “try it, and if it goes wrong, revert”. Here there is no revert, so all of the safety has to move in front of the change, into the window where turning back is still free.

pve8to9 is Proxmox’s own answer to that, written by the people who know what breaks, updated with every PVE 8 package update, and free to run. There is no argument against running it, and the only reason teams skip it is that most upgrades succeed without it — which is exactly the shape of habit that erodes until the run where it mattered.

The judgement it requires is small but real. FAIL is a stop, and nothing enforces that except the person running the window. WARN deserves investigation rather than dismissal, because the two that recur — an offline node and an inactive storage — are both failures with a delay. And FAILURES: 0 needs reading precisely: it clears the platform, which is one class of risk, and leaves guests, automation, backups and firmware entirely to you.

Used that way, the checker is the difference between an upgrade you decided to do and one you discovered you had started.