Before deploymentkubernetes-backup-dr-readiness

Backup and DR Readiness Checklist

12 items ·12 critical ·0 warn ·0 info

How to use this checklist

An untested backup is a hypothesis. The only items on this list that count are the ones where somebody has performed the restore.

Mark an item N/A when it genuinely does not apply, and write down why. An unexplained N/A is the most common way a checklist stops working.

Sign-off

Every critical item must pass. A failing critical item blocks the deployment or the maintenance window; it is not a note for later. Record the date, the reviewer, and the disposition of every item that did not pass.

Critical12 items

etcd snapshots are taken on a schedule that meets the RPO, and the schedule is monitored so that a silently failing job is noticed.
```
etcdctl snapshot save /backup/etcd-$(date +%F).db
```
etcd snapshots are copied off the control-plane nodes. A snapshot stored only on the node it protects is not a backup.
Snapshot integrity is verified after each run, not just the exit code of the command that produced it.
```
etcdutl snapshot status /backup/etcd-<date>.db --write-out=table
```
A full etcd restore has been rehearsed on a non-production cluster within the last quarter, and the elapsed time was measured against the RTO.
Workload state — PersistentVolume data — is backed up independently of etcd. Restoring etcd restores object definitions, not data.
Every workload manifest is reconstructible from Git, so that a rebuild does not depend on the etcd backup at all.
Secrets are recoverable from the external secret store or an encrypted backup, and the decryption key is itself recoverable.
A PersistentVolume restore for at least one stateful workload has been rehearsed end to end within the last quarter.
RTO and RPO are written down, agreed with the service owner, and consistent with the measured drill times rather than aspirational.
A rebuild-from-nothing runbook exists, names who does what, and has been read by someone who was not its author.
The DR plan accounts for dependencies outside the cluster: registry, DNS, identity provider, secret store, load balancer.
Backups are in a separate failure domain and a separate trust domain, so that a compromise of the cluster cannot delete them.