Purpose
This checklist triages a Terraform incident when a plan or apply has failed in production. Run through every item before the recovery actions.
Order of work
The items run in four phases and the frontmatter list is in that order.
incident-1 and incident-2 are the immediate phase: stop the apply and
identify the failed resource. incident-3 to incident-6 are the
investigation: the cause of the failure, the provider status page, the audit
log, and whether the state still matches the real world. incident-7 to
incident-9 are the recovery itself. incident-10 to incident-12 are the
documentation that closes the incident out.
Verification
The checklist is complete when every item is checked. A
critical item unchecked is a blocker.
Escalation
If a critical item cannot be satisfied, escalate to the incident commander or the engineering manager.