Live restore, single-file restore and the RTO arithmetic · Proxmox VE

An RTO is a promise about how long a service can be gone. It is usually written by someone who has never watched a 2 TiB restore run over a 1 GbE link, and it is usually a number with no arithmetic behind it.

This lesson supplies the arithmetic, and then supplies the two mechanisms that change the answer: live restore, which cuts time-to-service to minutes at a cost while it runs, and single-file restore, which avoids restoring the machine at all.

The baseline: what a full restore actually costs

Read-only / Saferestore time model - fill in your own measured numbers— Read-only arithmetic. The only input you should not guess is THROUGHPUT_MIBPS, which you get by timing a real restore.

set -euo pipefail

GUEST_GIB=512            # the size of the data to be written back
THROUGHPUT_MIBPS=110     # MEASURED restore throughput, not link speed
DECISION_MIN=15          # time to notice, decide and authorise
BOOT_MIN=4               # guest boot and service start
VERIFY_MIN=10            # someone confirms the service actually works

TRANSFER_MIN=$(( GUEST_GIB * 1024 / THROUGHPUT_MIBPS / 60 ))
TOTAL_MIN=$(( DECISION_MIN + TRANSFER_MIN + BOOT_MIN + VERIFY_MIN ))

printf 'transfer   %d min\n' "$TRANSFER_MIN"
printf 'total RTO  %d min (%d h %d m)\n' "$TOTAL_MIN" "$((TOTAL_MIN/60))" "$((TOTAL_MIN%60))"

At those numbers a 512 GiB guest is roughly 79 minutes of transfer and about 108 minutes of total outage. Three observations that change how people write RTOs:

Throughput is not link speed. A 1 GbE link tops out around 118 MiB/s in theory; a restore also has to read chunks off the datastore, decompress and decrypt them, and write them to the destination storage. Measure the number by restoring something, once, and record it. A restore plan built on the theoretical link speed is out by a factor you will discover during the incident.

The non-transfer terms are not small. Fifteen minutes of decision and ten of verification is 23% of the total here, and on a small guest they dominate completely. Restoring faster does not help an organisation whose bottleneck is finding someone authorised to say yes.

Parallel restores do not scale linearly. Restoring ten guests at once shares the same datastore read path and the same link. If your DR plan assumes ten simultaneous restores each at full speed, it assumes ten times the bandwidth you have.

Live restore

Service impact possiblelive restore - the guest starts immediately— PBS-backed backups only. The VM boots at once and the data arrives underneath it while it runs.

set -euo pipefail

# From a PBS-backed storage. The archive name is the snapshot path.
qmrestore 'pbs-main:backup/vm/214/2026-08-11T01:00:03Z' 214 \
--live-restore 1 \
--storage ceph-vm

# Restore alongside the original instead of over it, for a comparison or a
# partial recovery. --unique rerolls the MAC addresses so the clone does not
# collide with the original on the network.
qmrestore 'pbs-main:backup/vm/214/2026-08-11T01:00:03Z' 9214 \
--live-restore 1 \
--storage ceph-vm \
--unique 1

The documented behaviour:

Enabling live-restore via either the checkbox in the GUI or the --live-restore argument of qmrestore causes the VM to start as soon as the restore begins. Data is copied in the background, prioritizing chunks that the VM is actively accessing.

That last clause is the mechanism. It is not a naive background copy that happens to run while the guest is up — the restore follows the guest. When the guest reads a block that has not arrived yet, that chunk is fetched with priority. So the boot path, the service binaries and the working set arrive first, in the order the guest asks for them, and the cold remainder of the disk trickles in behind.

	Full restore	Live restore
Time to service	Full transfer + boot	Boot, essentially immediately
Performance during	N/A, nothing is running	Degraded reads; writes largely unaffected
Total data moved	Same	Same
If it fails part-way	Restore failed, nothing lost, retry	VM is in an undefined state and data written during the attempt likely cannot be preserved
Available for	Any backup storage	PBS only, VMs only

Reads are degraded and writes are not, which is not intuitive

The documentation notes that the VM operates with reduced disk read speeds during the restore, while writes are largely unaffected.

The asymmetry follows from the mechanism. A read may need a chunk that has not arrived, so it blocks on a fetch from PBS. A write goes to the destination storage directly — there is nothing to wait for, because the data being written is replacing whatever would have been restored there.

Which tells you what live restore is good and bad at:

Good: a service that boots, reads its working set once, and then mostly writes. Web servers, application servers, log collectors, most appliances.
Bad: anything read-heavy over a large cold dataset. A reporting database that scans historical tables will be pulling those tables over the network as it scans them, and will feel unusable for as long as that takes.

A failed live restore is not a no-op

An ordinary restore that fails leaves you where you started: the target was being written, the source snapshot is untouched, you fix the cause and run it again.

A live restore that fails is different, and the documentation is blunt about it: the VM is left in an undefined state, and data written during the failed attempt likely cannot be preserved.

That is the whole risk, and it is why live restore is a decision rather than a default. You started a production service on top of a disk image that was still arriving; if the arrival stops, the service has been running and writing against an incomplete disk. Those writes are not recoverable in any useful sense, and the guest is not in a state you can reason about.

The practical consequences:

Do not live-restore over the original VM ID if there is any chance the original disk still holds data you might want. Restore to a new ID.
Do not live-restore across a link you do not trust. A restore that has to survive an hour of WAN is exactly the one most likely to fail part-way.
Have the fallback decided in advance: if the live restore fails, the next step is a full restore to a fresh VM ID, and someone needs to know that before it happens.

Why live restore is PBS-only

A restore from a .vma file on a directory or NFS storage is a stream. The archive is read from the beginning to the end and written to the destination in that order. There is no way to ask it for the block the guest wants next, because the format does not support addressing an arbitrary block without walking to it.

A PBS snapshot is not a stream. It is an index of chunk digests, and the chunk store is content-addressed. Given an offset in the virtual disk, PBS can compute which chunk covers it, look up its digest in the index, and fetch exactly that chunk. Random access is a first-class operation.

That is the entire reason live restore works only against PBS, and it is the same structural property that makes single-file restore possible and makes deduplication possible. The chunk store is not just a space optimisation — it is what makes a backup randomly addressable, and random addressability is what turns a backup into something you can run a machine from.

Single-file restore: the case that is not a restore

The most common recovery request is not “the VM is gone”. It is “someone deleted a directory” or “we need the config file as it was on Tuesday”. A full restore for that is absurd, and a live restore only slightly less so.

The PVE web interface exposes a File Restore button on a PBS-backed backup, which lets you browse the snapshot and download individual files or directories. That is the right tool for a service-desk request and it needs no CLI.

From the command line, proxmox-backup-client gives two paths into a snapshot without materialising a whole guest:

Read-only / Safebrowse a snapshot interactively— Read-only against the datastore. The catalog shell navigates the archive and selects what to pull out.

set -euo pipefail
export PBS_REPOSITORY='restore@pbs@pbs1.example.com:store1'

proxmox-backup-client snapshot list

proxmox-backup-client catalog shell 'host/app01/2026-08-11T01:15:00Z' root.pxar
# Inside the shell: ls, cd, pwd, find, list-selected, restore-selected, restore

Read-only / Safemount a snapshot and copy out of it— Mounts the archive read-only via FUSE. Nothing is written to the datastore; unmount when finished.

set -euo pipefail
export PBS_REPOSITORY='restore@pbs@pbs1.example.com:store1'

mkdir -p /mnt/restore
proxmox-backup-client mount 'host/app01/2026-08-11T01:15:00Z' root.pxar /mnt/restore

cp -a /mnt/restore/etc/nginx/nginx.conf /root/nginx.conf.tuesday

umount /mnt/restore

Destructiverestore a whole archive to a directory— Writes to the target path. Restore to an empty staging directory and move files into place deliberately, never straight over a live tree.

proxmox-backup-client restore 'host/app01/2026-08-11T01:15:00Z' root.pxar /srv/staging/app01-restore/

Choosing, under pressure

Situation	Path	Why
One file or directory deleted	Single-file restore	Minutes, no outage, no rollback of anything else
Guest corrupted, service must be back now, workload is write-heavy	Live restore to a new VM ID	Time-to-service in minutes, degradation is tolerable, original preserved
Guest corrupted, workload is a large read-heavy dataset	Full restore	Live restore would be unusable for the duration and the transfer time is the same either way
Restore across a WAN or an unreliable link	Full restore	A failed live restore leaves an undefined guest; a failed full restore leaves nothing
Whole cluster lost	Full restore, staged by priority	Bandwidth is the constraint. Restore in dependency order and accept a queue
You need to compare current state against a backup	Live restore or full restore to a new ID, `--unique`	Never restore over the thing you are still investigating

What happens if you have never rehearsed the path you choose?

You discover the prerequisites during the incident, and they are not small.

You do not know your throughput number. So the RTO you quote to the incident channel is a guess, and it is usually optimistic by a factor of two or more. People make staffing and communication decisions on that number.
You do not know whether restore permissions work. The account you use day to day may have DatastoreBackup — which permits backing up and restoring owned backups — and not the read access needed for the guest you are trying to recover. That is a five-minute problem on a Tuesday and a forty-minute problem at 03:00 when the person who can grant it is asleep.
You do not know the snapshot naming. qmrestore wants a specific archive path. Finding it under pressure, from the CLI, on a node whose GUI is not reachable, is a skill and not a lookup.
You have not seen the failure mode. The first time you watch a live restore fail should not be the time it matters.

The remedy is the restore certification drill covered in xiii-pbs-restore-practice, extended with these three paths. Restore one guest with --live-restore to a spare VM ID, time it, note the degradation, and write the throughput number in the runbook. It costs an hour a quarter.

Knowledge check

Why this matters

Restore is the only part of a backup system that has a customer. Everything else — chunk stores, prune schedules, verify jobs — exists to make this one operation possible, and it is the one part most estates have never actually performed under time pressure.

The habit worth building is that the shape of the restore is a decision with inputs. How much data, how fast can you actually move it, is the workload read-heavy or write-heavy, is the original disk still evidence, how reliable is the path, and what is the fallback if the clever option fails half-way. An operator who works through those five in ninety seconds recovers faster than one who reaches for whichever restore they have done before.

And the arithmetic has a second audience. The number that comes out of that script is the honest RTO, and handing it to the person who wrote the SLA is usually more valuable than any tuning you could do to shorten it.