Spotting State and Identity Changes

A Terraform PR that touches state is the highest-stakes PR in the engineering queue. The plan file is gone after the apply; the state file persists. A misapplied state change can leave the configuration and the cloud in agreement about a state that has no corresponding infrastructure, or about an infrastructure that has no corresponding state.

State-touching PRs are rare. They are reviewed with the same rubric as any other PR, plus four additional checks specific to the state. The reviewer walks the state checklist on every PR that includes a state mv, a state rm, an import block, a terraform import command, or a moved block that is non-trivial.

The state-mutating operations

Three commands and one block change the state. Each has a distinct purpose and a distinct review posture.

terraform state mv

state mv renames a resource in the state. The real world is unaffected; the address the configuration references after the rename is the same real-world object it was before. The state file is the only thing that changes.

terraform state mv aws_instance.legacy aws_instance.app

The legitimate uses:

• Refactoring a module structure. Moving a resource from module.legacy.aws_instance.web to module.app.aws_instance.web without recreating the instance.
• Renaming an instance key in a count / for_each block. Changing aws_instance.web[0] to aws_instance.web["prod"].
• Recovering from a typo in the resource name that has not yet been merged.

The illegitimate uses:

• “I want to delete this resource from Terraform but not from the cloud.” That is state rm, not state mv.
• “I want to fix drift by mutating the state.” That is a misread of the drift; the fix is to refresh the state or to fix the cloud, not to edit the state.

terraform state rm

state rm removes a resource from the state. The real world is unaffected; the configuration’s next plan proposes to recreate the resource from the cloud’s perspective as a new object.

terraform state rm aws_security_group.legacy

The legitimate uses:

• Removing a resource from Terraform management that was imported into the configuration for one-off management but is now managed elsewhere. The next plan will propose to recreate the resource; the operator prevents the recreate by removing it from the state.
• Splitting state files. When a workspace’s state is being split into multiple workspaces, resources that will live in the new state are removed from the old state’s resources list, then imported into the new.
• Recovery from a corrupt state entry. A resource has an unreadable state entry; the operator removes the entry, then re-imports the resource from the cloud.

The illegitimate uses:

• “I want to delete this resource.” That is terraform destroy, not state rm. state rm does not delete the real-world object.
• “I want to fix the state because the resource does not exist in the cloud.” Removing the resource from the state causes the next plan to recreate it. If recreation is wrong, the resource must be fixed in the cloud, not in the state.

terraform import and import blocks

terraform import brings an existing real-world object under Terraform’s management. The object exists in the cloud; the state records its existence and attributes; the configuration describes what the object should look like; the next plan checks that the description matches the object.

terraform import aws_security_group.adopted sg-0abc123def456789

aws_security_group.adopted: Importing from ID "sg-0abc123def456789"...
aws_security_group.adopted: Import prepared!
  Prepared aws_security_group for import
aws_security_group.adopted: Refreshing state...
aws_security_group.adopted: Import complete!

  Resource must be updated in-place
  Resource is unchanged after the update.

import blocks (Terraform 1.5+) let you write the import in the configuration so the same import is reproducible:

import {
  to = aws_security_group.adopted
  id = "sg-0abc123def456789"
}

resource "aws_security_group" "adopted" {
  name        = "adopted"
  description = "Adopted from console management"
  # ...the rest of the configuration
}

The legitimate uses:

• Adopting existing infrastructure. A workspace is created for an environment that already has resources; the resources are imported into the configuration as the first step.
• Recovering from a state rm. A resource that was removed from the state is re-imported if the real-world object is desired.
• Splitting state. When a state file is split, each resource is imported into the new state file.

The illegitimate uses:

• “I want to re-import because the resource drifted.” Drift is fixed by terraform plan -refresh-only, not by import.
• “I want to overwrite the state with a different ID.” Import does not overwrite; it appends. The right tool is state mv if the same address has multiple state entries.

moved blocks

moved blocks (Terraform 1.1+) tell Terraform that a resource address has been changed in the configuration without recreating the real-world object. The block appears in the configuration; the state is updated by the next plan or apply; the real world is unchanged.

moved {
  from = aws_instance.legacy
  to   = aws_instance.app
}

The next plan shows:

aws_instance.app: Refreshing state...
aws_instance.app: Import complete!

  # aws_instance.legacy has moved to aws_instance.app
  ~ resource "aws_instance" "legacy" {
      id = "i-0abc123def456789"
    }

The plan contains no ~, no +, no -; only the moved notice. The state is rewritten to point the new address to the existing real-world object.

moved blocks are the right tool when the rename is part of normal refactoring and the rename should be applied by every workspace that consumes the configuration. state mv is the right tool when the rename is local to one state file and should not be committed to the configuration.

The audit trail that every state PR must include

A state-touching PR is the only PR class that requires the reviewer to look at the state file directly. The reviewer verifies four artefacts in the PR before approving.

1. A state backup taken before the change

The state file is downloaded from the backend and backed up to a known location with a timestamped name.

# Before any state-touching PR
terraform state pull > state.bak.$(date -u +%Y%m%dT%H%M%SZ).json

-rw-r--r-- 1 ops ops 124518 Aug 13 11:24 state.bak.20260813T112400Z.json

The backup filename is recorded in the PR description. The backup is uploaded to the artifact store with the retention of the audit window. The backup is the rollback target.

2. A state list before and after

The reviewer compares the resource list before the change and after the change.

terraform state list > before.txt
# apply the state-touching change
terraform state list > after.txt
diff before.txt after.txt

A state mv produces a list with one address changed. A state rm produces a list with one address removed. An import produces a list with one address added. A moved block produces no list change (the state internals change; the list does not).

The diff is included in the PR comment.

3. A state show of the changed resource

The reviewer reads the resource’s full state entry before and after.

terraform state show aws_security_group.legacy \
    > before.json
terraform state mv aws_security_group.legacy \
    aws_security_group.adopted
terraform state show aws_security_group.adopted \
    > after.json
diff before.json after.json

For a state mv, the diff is only the address. For an import, the diff is the attributes that the cloud returned. For a state rm, the resource is gone from the after list.

4. The justification

The PR description names why the state change is necessary. The justification is specific:

Justification: Renaming aws_instance.legacy to
aws_instance.app. The legacy address was a typo
from a previous refactor; the production instance
is at aws_instance.app. The rename aligns the
state with the configuration. Ticket REL-1042.

NOT a justification: "Cleaning up state."

The justification must include the ticket or incident reference. The reviewer verifies the ticket exists in the issue tracker and that the ticket calls for the state change.

The right rollback

State changes are reversible in three ways, in order of preference.

1. Push the backup state file back

The simplest rollback. The state file before the change is restored; the configuration is unchanged; the next plan will propose the original state.

# Push the backup back via the backend's API
terraform state push state.bak.20260813T112400Z.json

Acquiring state lock. This may take a few moments.
The state will be pushed.
# ...
State push successful.

For S3+GCS backends, the equivalent is to upload the backup file directly to the state object and unlock. For local backends, the equivalent is terraform state push <backup>.

The push is the standard rollback for state mv and moved blocks. It is also the right rollback for import where the import added a resource that should not have been added.

2. Run the inverse state operation

For state rm, the inverse is state mv back to the original address. For terraform import, the inverse is terraform state rm of the imported address.

# Inverse of: terraform state rm aws_sg.legacy
terraform import aws_sg.legacy sg-0abc123def456789

The inverse is appropriate when the backup is not available (the backup was deleted, the backup is corrupt) or when the change is small enough that the inverse is faster than the push.

3. Recreate the real-world object

If neither the backup nor the inverse is available, the only remaining rollback is to destroy the real-world object and let the next plan recreate it. This is the worst-case rollback for an import of a resource that should not have been adopted; the real-world object must be deleted, the state is empty, and the next apply creates the resource fresh.

Recreation is a destructive rollback. It is reserved for situations where the state is irrecoverably corrupted and the real-world object is not. The rollback requires explicit sign-off from a senior engineer and a change ticket that documents the recreation.

Verifying the state-touching PR

The PR is verified by three checks.

1. The plan after the change is the expected plan

terraform plan -detailed-exitcode -out=tfplan
terraform show tfplan | tail -5

No changes. Your infrastructure matches the configuration.

For a state mv or moved block, the next plan should be empty (or show only the desired changes). The reviewer confirms the plan matches the ticket’s “after” state.

2. The state list matches the inventory

The inventory is a separate record (typically a Spreadsheet, CMDB, or in-code reference) of resources managed by each workspace. The state list should match the inventory after the change.

terraform state list | sort > state.txt
diff -u inventory.txt state.txt

A diff with new entries (from import) or missing entries (from state rm) is expected. A diff with unrelated changes is drift; the reviewer rejects.

3. The cloud reality matches the state

A sampled comparison of the live cloud against the state for a few key resources catches the case where the import found a different real-world object than the configuration intended.

terraform state show aws_security_group.adopted \
    | jq '.attributes | {id, name, description}'

aws ec2 describe-security-groups \
    --group-ids sg-0abc123def456789 \
    --query 'SecurityGroups[0].{GroupName:GroupName,Description:Description}'

The two should match. A mismatch is a signal that the import captured the wrong object; the rollback is to push the backup and start over.

Production failure modes

1. state rm followed by an apply that recreated the resource from scratch. The author removed the resource because they believed the cloud did not contain it. The cloud did contain it; the next plan recreated it empty. The fix is to verify the cloud before state rm; the state rm is appropriate only when the real-world object is being deleted elsewhere or is already gone.

2. state mv run on production without a backup. The state file became inconsistent; the next apply failed; the rollback required pulling the state from the backend’s versioned history (S3 versioning, GCS object versioning) rather than a known good backup. The fix is the backup step in the PR template; merge is blocked until the backup filename is in the description.

3. terraform import on the wrong ID. The author imported sg-0abc123def456789 instead of sg-0def456789abc123. The state now references the wrong SG. The fix is the state-show check before the import is committed; the diff between the desired and imported attributes catches the typo.

4. moved block applied to a non-trivial rename with side effects. The moved block renames the address; the configuration still references the new address; a downstream aws_lb_target_group_attachment that referenced the old address now sees a missing dependency. The fix is to update the downstream references in the same PR.

5. state rm used to “fix” drift. The operator ran state rm to remove a resource that drifted. The drift remained; the next plan proposed to recreate the resource. The fix is terraform apply -refresh-only and then plan; do not edit the state to fix drift.

6. No rollback in the PR description. The state was corrupted by the change; the operator did not know how to roll back. The fix is the PR template field for the rollback command.

Security and performance

Security: a state file pulled to a developer laptop is sensitive; the backup file is the same. The backup is uploaded to the artifact store with the same access control as the state file. It is not committed to the repository.

Performance: the state commands are fast (milliseconds to seconds). The bottleneck is the backup and the verification; both are one-line scripts in a typical state backend.

Production guidance

• The PR template requires a backup filename, a state list diff, and a rollback command. The merge is blocked until all three are in the description.
• A state mv for non-trivial renames is documented as a moved block in the configuration, not as a CLI command. The CLI is for one-off state fixes.
• state rm is rare; the right pattern is to delete the resource from the configuration and let the next apply destroy it.
• terraform import is preceded by a state-show of the real-world object and followed by a state-show of the imported state. The diff between the two is the review artefact.
• The backup is uploaded to the artifact store. It is not on a developer laptop and not in /tmp on a runner after the job.

What comes next

This lesson closes the Part XVI plan-review sequence. The next part covers incidents: how a production sysadmin recognises, investigates, and recovers from the Terraform-related incidents a careful plan review aims to prevent.

Verification

Take a state-touching PR from the team’s history (or construct one against a throwaway account). Walk the audit-trail checklist: backup filename in the PR, state list diff in the comment, state show diff for the changed resource, justification that references a ticket. Note the gaps; those are the gaps in the team’s discipline.