Skip to main content
RunBook Academy

← All checklists in Backup & DR

Quarterlybdr-backup-readiness

Backup production readiness

20 items ·14 critical ·6 warn ·0 info

How to use

Quarterly, one system per pass, one row per system in the recovery inventory. The order is deliberate. Items one to ten establish that a copy exists, sits where the original disaster cannot reach it, and can be opened. Items eleven to twenty establish that somebody notices when it stops, has read it back, and timed the reading. A system that passes the first half and fails the second is not protected; it is copied, which is a weaker claim and a different one.

Work from artefacts, not recollection. Each item names the thing that settles it — a listing, a dated run, an access policy, a stopwatch reading. If the artefact cannot be produced inside the hour, the item fails, and it fails today rather than during the incident that would otherwise have discovered it.

Where the numbers come from

Three things on this list are measurements; the rest are decisions. Confusing the two is how an estate acquires a recovery plan it cannot execute.

Recovery point age is read from the repository listing, never from the job log, because the two disagree exactly when it matters.

The verification date belongs to a run that read the stored data, which is not what a structural check does:

Read-only / Safea structure-only check, run on a repository with a corrupted data pack
$ restic check
using temporary cache in /tmp/restic-check-cache-2185980449
create exclusive lock for repository
load indexes
check all packs
check snapshots, trees and blobs
[0:00] 100.00%  2 / 2 snapshots
no errors were found

Exit code 0, and the next restore from that repository lost a file. Run against the same repository moments later, restic check --read-data named the damaged pack and exited 1. The date this item wants is the date of the run that read the data; the structural pass is a different measurement of a different thing.

The restore duration is a stopwatch reading taken during a real restore onto infrastructure that is not the source, split by phase and stopped when the application returns correct results rather than when the copy finishes.

Retention depth and the published recovery time objective are decisions, and each needs its basis recorded beside it: the period a regulator requires, the interval this estate has actually observed between damage and discovery, and the measured restore plus the overheads the drill left out.

Access this needs

Read access to the repository or storage account, to its retention and lifecycle configuration, and to the access policy that says who may delete from it — the last of those is what turns the fourth item from an opinion into an enumeration. Read access to the alerting configuration, and the escrow custodian’s contact so the second key can be confirmed to exist without being handed over.

Non-production capacity to restore onto: a host, a volume or an account that is not the source. Half this list cannot be completed without it, and discovering during the review that none can be arranged is itself the finding. Prefix anything created for the review rbdr- so it can be located and removed afterwards.

The reviewer should not be the person who built the backup.

What the review produces

One dated row per system: the failure models claimed, the copy standing behind each one, the identities able to delete those copies, the retention numbers with their basis, the date of the last data-reading verification and of the last restore onto other infrastructure, the measured duration of that restore, and the published objective with the arithmetic connecting it to the measurement.

Every finding carries an owner and a date. A system whose row cannot be completed is reported to its service owner as an unprotected system, in those words: “backed up” was the claim under test, and it did not survive the hour.

Sign-off

  • Date: ____
  • Reviewer: ____
  • System owner: ____
  • Service owner: ____

Every critical item must pass. A failing critical item is a blocker with a named owner, not a note for next quarter. Record the disposition of every item that did not pass, and the name of whoever accepted the residual risk.

Critical14 items

  1. restic snapshots
  2. restic key list
  3. restic snapshots
  4. restic check --read-data

Warning6 items

  1. restic forget --keep-daily 7 --keep-weekly 4 --dry-run