How to use
Run this once, on the recovered system, after the restore has finished and before anybody announces that the service is back. It is not a readiness review and it does not repeat quarterly; it belongs to one incident and it closes with that incident.
Work it top to bottom without reordering. The sequence is deliberate: bytes first, then the meaning of those bytes, then the point in time they represent, then the surrounding services, then the path a user takes, then the instrumentation that will tell you about the next failure. Every step depends on the ones above it, and an application transaction that succeeds says nothing useful while the file-level verification is still unrun.
Two questions run underneath the whole list. Is the recovered system correct, or is it merely running? And can the answer be shown to somebody who was not in the room? An item passes on an artefact — a status, a path, a number, a timestamp — and never on an operator’s recollection of a terminal that has since been closed.
Where the numbers come from
The verification items come from a measured run. A restore whose per-file comparison succeeds looks like this, and the second exit status is the one that carries the evidence.
$ restic restore 3fe43af4 --target /work/restore && md5sum -c manifest.md5restoring snapshot 3fe43af4 of [/work/prod] at 2026-08-28 13:27:02.65376235 +0000 UTC by root@8211a08b55c3 to /work/restore
Summary: Restored 7 files/dirs (60.000 MiB) in 0:00
>>> exit code: 0
--- comparing the restored tree against the 09:00 checksums ---
./app/app.conf: OK
./app/orders.csv: OK
./db/data.bin: OK
>>> md5sum -c exit code: 0Note what the first line supplies for free: the recovery point actually being restored, printed by the restore itself. That is the value the recovery-point items ask for, and reading it costs nothing.
The point-in-time items come from a separate capture on PostgreSQL 18.6, where
a base backup was rolled forward past an unqualified DELETE to a chosen
target time.
$ pg_ctl -D /work/pitr start # recovery_target_time set to 2026-08-28 13:34:40.077562+00 2026-08-28 13:35:12.708 UTC [631] LOG: starting point-in-time recovery to 2026-08-28 13:34:40.077562+00
2026-08-28 13:35:12.708 UTC [631] LOG: redo starts at 0/3000028
2026-08-28 13:35:12.713 UTC [631] LOG: restored log file "000000010000000000000004" from archive
2026-08-28 13:35:12.713 UTC [631] LOG: completed backup recovery with redo LSN 0/3000028 and end LSN 0/3000120
2026-08-28 13:35:12.713 UTC [631] LOG: consistent recovery state reached at 0/3000120
2026-08-28 13:35:12.713 UTC [625] LOG: database system is ready to accept read-only connections
2026-08-28 13:35:12.726 UTC [631] LOG: restored log file "000000010000000000000005" from archive
2026-08-28 13:35:12.735 UTC [631] LOG: recovery stopping before commit of transaction 836, time 2026-08-28 13:34:42.096745+00
2026-08-28 13:35:12.735 UTC [631] LOG: redo done at 0/52EBC90 system usage: CPU: user: 0.01 s, system: 0.00 s, elapsed: 0.02 s
2026-08-28 13:35:12.735 UTC [631] LOG: last completed transaction was at log time 2026-08-28 13:34:38.041366+00Three distinct instants appear there: the one requested, the one recovery
stopped before, and the one actually held. They differ by seconds here and can
differ by much more. The recovery was accepted because the recovered database
returned rows recovered : 50000 (expected 50000) and
sum(amount) : 825025000 (expected 825025000) — a property of the
business data recorded before the incident, not the observation that the server
had started.
Elapsed times come from timestamps written as each stage ended. Nothing on this list is estimated afterwards.
Access this needs
Read access to the recovered filesystem and to the backup-time manifest, from the recovery host. Read access to the recovery log of whatever performed the restore, and to the incident record where the statuses were captured.
Database or application credentials sufficient to evaluate the business invariant and to execute one write transaction end to end, plus agreement from the service owner that a synthetic record may be created and then removed.
Sight of the monitoring configuration and the standing to trigger one deliberate failure. Write access to the runbook repository, because the last item is a commit.
No item requires write access to the backup repository.
What the review produces
A dated section appended to the incident record, naming the operator, listing every item and its disposition. Six values belong at the top: the restore exit status, the verification exit status, the recovery point actually reached, the business invariant with its expected and observed values, the work lost behind the recovery point, and the total elapsed time with its per-stage breakdown.
Two of those leave the engineering team. The recovery point and the work lost behind it go to the named service owner in writing before the incident closes. The measured elapsed time goes to whoever owns the recovery time objective, whether or not it is comfortable reading.
One artefact stays: the runbook commit. An incident that ends without one has converted a day of expensive learning into a story.
Sign-off
- Reviewer: ____
- Date: ____
- Service owner (recovery point and lost work accepted): ____
- Runbook commit: ____
Every critical item passes, or the recovered system stays out of service until a named owner accepts the residual risk in writing with a remediation date.