Skip to main content
RunBook Academy

TerraformXXI · Testing, Linting, and Static AnalysisProduction Terraform

The Layered Testing Model

Intermediate⏱ ~12 min

What you'll learn

  • Apply the layered testing model
  • Identify the layer boundaries
  • Choose the right test

Prerequisites

None — start here.

Verified against Terraform CLI 1.9.x · OpenTofu 1.7.x · HCL 2.0 · bpg/proxmox provider 0.66+ · hashicorp/local provider 2.5+ · hashicorp/null provider 3.2+ · hashicorp/random provider 3.6+ · hashicorp/http provider 3.4+ · Ubuntu 24.04 LTS · Debian 12 (Bookworm)

Not yet marked complete on this device.

Objective

Apply the layered testing model

What this lesson covers

  • Identify the layer boundaries
  • Choose the right test

Why this matters in production

Production Terraform operations have blast radius. The Layered Testing Model is one of the operational controls that determines whether a change is safe to apply. Skip this lesson at the cost of not understanding a production control.

How it works

Provide an overview of the topic. This lesson covers the the layered testing model concept, the underlying mechanism, and the production implications.

Key concepts.

  • The terminology used in the topic.
  • The mechanism that determines the operational behaviour.
  • The production control that mitigates the operational risk.

Operational implications.

The the layered testing model concept affects the production change-management workflow. A change in this area has blast radius across the entire estate.

How to configure it

Apply the configuration pattern:

  1. Set up the configuration block.
  2. Validate the configuration.
  3. Run the plan.
  4. Verify the operations.
  5. Document the change.

How to inspect it

Inspection pattern:

  • Use the relevant CLI command to inspect the state.
  • Verify the configuration matches the expectation.
  • Confirm the plan is empty.

Validation

The validation pattern:

  • Confirm the configuration is valid.
  • Verify the plan is empty.
  • Confirm the operations are correct.

Production failure modes

The following failure modes are the most common in production:

  1. Misconfiguration of the relevant parameter.
  2. Drift between the configuration and the real world.
  3. Provider failure during the apply.
  4. State corruption or loss.

How to recover

The recovery procedure:

  1. Investigate the failure.
  2. Identify the cause.
  3. Apply the remediation.
  4. Verify the state.
  5. Document the incident.

The following runbooks in this course apply to this lesson:

  • terraform-runbook-investigate-state-lock
  • terraform-runbook-recover-partial-apply
  • terraform-runbook-investigate-provider-failure

What comes next

The next lesson in XXI-Testing continues the exploration of The Layered Testing Model. Follow the order to maintain the production-first narrative.

Knowledge check · 7 questions

  1. Q1. What is the role of the layered testing model?

  2. Q2. What is terraform test?

  3. Q3. terraform test is a separate tool that requires installation.

  4. Q4. What is the role of mock providers in tests?

  5. Q5. Which of the following are part of the layered testing model? (Select all that apply.)

  6. Q6. What is the role of the plan test in terraform test?

  7. Q7. A terraform test fails. What is the next step?

Passing score: 75%. Answers are checked in this browser.