Backend Migration Done Right

A backend in Terraform is where the state file lives and how locking is enforced. The first backend most teams use is the local one: a file on the engineer’s laptop or on a single build host. That backend has no locking, no versioning, no shared access, and no audit trail. Production teams outgrow it within weeks.

This lesson covers the procedure for moving from the local backend to S3 with DynamoDB locking, for upgrading an existing S3-only backend to add locking, and for partial overrides where one credential changes but the rest of the backend stays the same. The procedure is short. The preconditions are the part that takes the time.

Backend versus state: the terminology

The two terms are often used interchangeably and they are not the same.

• Backend is the configuration in the terraform { backend ... } block. It declares where state lives and how it is locked.
• State is the JSON document that Terraform reads and writes. A migration of state moves the document; a migration of backend changes the configuration that says where the document should live.

Most production migrations are backend migrations that include a state migration as a side effect. When you change the backend block from local to s3, the state moves from the local file to S3. The state is not the primary thing being changed; the backend is.

The four migrations teams actually do

Not every team follows the same path. The procedure varies by source and destination:

  Source            Destination            Migration class
  ---------------   --------------------   ----------------------
  local             S3 (no lock)           New backend, no locking
  local             S3 + DynamoDB lock     New backend, with locking
  S3 (no lock)      S3 + DynamoDB lock     Add locking, same bucket
  S3 (region A)     S3 (region B)          Same tooling, new region
  any               Terraform Cloud        New tooling, new host

The third row is the one most teams get wrong. They already have the state in S3, so the migration feels trivial. In practice, adding a lock table means introducing a new IAM permission and a new race window between the lock write and the state read.

Pre-flight: what must be true before init

Three resources must exist before terraform init -migrate-state is run. None of them are created by the migration; all of them are read by it.

# 1. S3 bucket with versioning on.
resource "aws_s3_bucket" "tfstate" {
  bucket = "acme-tfstate-prod"
}

resource "aws_s3_bucket_versioning" "tfstate" {
  bucket = aws_s3_bucket.tfstate.id
  versioning_configuration {
    status = "Enabled"
  }
}

# 2. DynamoDB table for locking. The partition key is fixed:
# Terraform writes the lock ID as a string attribute named
# LockID.
resource "aws_dynamodb_table" "tfstate_lock" {
  name         = "acme-tfstate-lock"
  billing_mode = "PAY_PER_REQUEST"
  hash_key     = "LockID"

  attribute {
    name = "LockID"
    type = "S"
  }
}

# 3. IAM policy granting the Terraform role access to both.
data "aws_iam_policy_document" "tfstate" {
  statement {
    sid    = "StateBucket"
    effect = "Allow"
    actions = [
      "s3:ListBucket",
      "s3:GetObject",
      "s3:PutObject",
      "s3:DeleteObject",
    ]
    resources = [
      aws_s3_bucket.tfstate.arn,
      "${aws_s3_bucket.tfstate.arn}/*",
    ]
  }

  statement {
    sid       = "StateLock"
    effect    = "Allow"
    actions   = [
      "dynamodb:GetItem",
      "dynamodb:PutItem",
      "dynamodb:DeleteItem",
    ]
    resources = [aws_dynamodb_table.tfstate_lock.arn]
  }
}

The DynamoDB lock attribute must be named LockID. Terraform’s S3 backend uses that exact name as the partition key; any other name means lock acquisition always fails.

# READ-ONLY: confirm the preconditions are met.
aws s3api get-bucket-versioning \
    --bucket acme-tfstate-prod \
    --query 'Status' --output text
aws dynamodb describe-table \
    --table-name acme-tfstate-lock \
    --query 'Table.[TableStatus,KeySchema[0].AttributeName]' \
    --output text

Enabled
ACTIVE    LockID

If the table status is anything other than ACTIVE, or the key attribute is not LockID, the migration does not start.

Procedure: local to S3 with DynamoDB

Step 1 — update the configuration:

# versions.tf
terraform {
  required_version = ">= 1.9.0"

  backend "s3" {
    bucket         = "acme-tfstate-prod"
    key            = "global/terraform.tfstate"
    region         = "eu-west-1"
    dynamodb_table = "acme-tfstate-lock"
    encrypt        = true
  }
}

Step 2 — run init and answer the prompt:

# CONFIGURATION: change the backend, copy the state, accept
# the prompt.
terraform init -migrate-state

Initializing the backend...
Backend configuration changed!

Terraform has detected that the configuration specified for the
backend has changed. Terraform will now check for existing state
in the new backend.

Do you want to copy existing state to the new backend?
  Pre-existing state was found while migrating the previous
  "local" backend to the newly configured "s3" backend.
  No existing state was found in the newly configured "s3" backend.
  Do you want to copy this state to the new "s3" backend?
  Enter "yes" or "no".

  Enter a value: yes

Successfully configured the backend "s3"!
Terraform will now use "s3" as the default backend.

Step 3 — verify the empty plan:

# READ-ONLY: confirm the migrated state matches reality.
terraform plan -no-color -detailed-exitcode
echo "exit=$?"

No changes. Your infrastructure matches the configuration.
exit=0

Step 4 — verify the lock is acquirable:

# CONFIGURATION: a no-op apply with locking left at its
# default exercises the lock acquire and release.
terraform apply -auto-approve

Locking is on by default, and that is the point of the check. A no-op apply reads the state from the bucket, acquires the lock, finds nothing to do, and releases the lock again, so it exercises the bucket read and every lock-table permission before a real change depends on them. If the table is unreachable, or the IAM policy is missing dynamodb:GetItem, dynamodb:PutItem, or dynamodb:DeleteItem, the command fails immediately and the migration can be aborted.

That proves the lock can be taken and released by one caller. To prove it also excludes a second caller, hold the lock and watch the second attempt refuse. In one shell:

# CONFIGURATION: holds the lock until the prompt is answered.
terraform apply

Leave that shell sitting at the approval prompt. In a second shell, against the same bucket and the same key:

# READ-ONLY: expected to fail while the first shell holds the lock.
terraform plan

plan takes the same lock apply does, so the second command must refuse rather than proceed:

╷
│ Error: Error acquiring the state lock
│
│ Error message: ConditionalCheckFailedException: The conditional
│ request failed
│ Lock Info:
│   ID:        8f3c1a04-6d2e-4b77-9c15-2ab7f0e4d913
│   Path:      acme-tfstate-prod/global/terraform.tfstate
│   Operation: OperationTypeApply
│   Who:       alice@build-01
│   Version:   1.9.5
│   Created:   2026-08-18 09:41:22.113272 +0000 UTC
│   Info:
╵

That refusal is the evidence. If the second command instead runs a plan, locking is not working, and the usual causes are a backend block with no dynamodb_table, two shells pointed at different key values, or a table name that is not the one the backend declares. Answer no at the first shell’s prompt to release the lock once the check is done.

Procedure: adding DynamoDB locking to an S3-only backend

The state is already in S3. The change is the backend block, the IAM policy, and the lock table:

# Before
backend "s3" {
  bucket = "acme-tfstate-prod"
  key    = "global/terraform.tfstate"
  region = "eu-west-1"
}

# After
backend "s3" {
  bucket         = "acme-tfstate-prod"
  key            = "global/terraform.tfstate"
  region         = "eu-west-1"
  dynamodb_table = "acme-tfstate-lock"
  encrypt        = true
}

# CONFIGURATION: reconfigure the backend with locking.
terraform init -migrate-state

The state does not need to move because it is already in S3, in the same bucket, at the same key. The CLI detects that the state is already present at the destination and skips the copy. The lock table is the only thing that has changed.

Partial override with -reconfigure

Sometimes one credential changes and the rest of the backend stays the same. A new assume-role ARN, a new region, a new endpoint. The full migration prompt is the wrong tool because it copies state that has not moved.

The flag is -reconfigure:

# CONFIGURATION: discard the cached backend configuration
# and re-initialise without prompting for a state copy.
terraform init -reconfigure

-reconfigure tells Terraform to forget the stored backend configuration and use the values from the configuration file as-is. The state is not touched. The lock is re-acquired on the next operation. This is the right tool when the bucket and key are unchanged but a credential or region has rotated.

For Terraform 1.1 and later, partial configuration sources (backend "s3" {} with no inline arguments and a backend.hcl or backend.tfbackend partial file) provide a cleaner path: pass the new partial on the command line and -reconfigure again. The state does not move because the bucket and key are unchanged.

Switching providers mid-migration

A common production case is the backend migration as part of a wider provider change. A team adopts the AWS provider 5.x, which requires Terraform 1.5 or later, while also moving from local to S3. The two changes are independent in code but coupled in the apply.

The order matters:

1. Bump the required_version and provider version constraints in versions.tf.
2. Run terraform init -upgrade to install the new provider.
3. Verify terraform plan against the existing state.
4. Apply the provider changes (this may show drift; review).
5. Update the backend block.
6. Run terraform init -migrate-state.

If steps 1 to 4 are skipped, the backend migration runs against the old provider, and the new provider is installed against the old backend. Either order works mechanically, but the migrations are easier to debug when the backend is the last change.

Production failure modes

1. Versioning is off on the destination bucket. A partial write corrupts the state. Symptom: subsequent plans fail with state file is corrupt and there is no version to restore.

2. The DynamoDB table uses a different partition key name. Terraform always writes to LockID. Symptom: lock acquisition fails with ResourceNotFoundException or silently creates orphan items.

3. Two operators run init concurrently. The state copy races and produces a corrupted state. Symptom: one of the two operators sees a successful migration; the other sees a checksum error on the next plan.

4. The IAM policy is missing one of the lock actions. State writes succeed; lock acquisition fails. Symptom: terraform plan works on a single host and fails for any second host or CI runner.

5. The backend block is moved to a separate file without updating .gitignore. The state file is committed to the version control system by accident. Symptom: the state appears in the next pull request review, including any sensitive output values it contains.

6. -reconfigure is used during a state copy. The flag skips the copy prompt and the state at the destination is stale or empty. Symptom: the next plan shows every resource as needing to be created.

What to do in production

• Enable versioning on every state bucket before the first state file is written.
• One DynamoDB lock table serves as many state files as you like. Terraform derives the LockID item from the bucket and the key, so two configurations with different keys hold independent locks in the same table and run concurrently. Split the table only when you want a separate blast radius or a separate IAM boundary, not because you think you have to.
• Gate the migration window with an out-of-band signal so only one operator runs init at a time.
• Run a no-op apply after the migration, with locking left on, to prove the lock is acquirable end to end.
• Treat the IAM policy as part of the migration. The state bucket access and the lock table access are separate statements in the policy; both must be present.
• Keep the old backend alive for at least one full change window after the migration. Decommission is the last step.

Verification

A backend migration is verified by four checks:

# READ-ONLY: confirm versioning is on.
aws s3api get-bucket-versioning \
    --bucket acme-tfstate-prod \
    --query 'Status' --output text

Enabled

# READ-ONLY: confirm the lock table is ACTIVE and uses
# LockID as the partition key.
aws dynamodb describe-table \
    --table-name acme-tfstate-lock \
    --query 'Table.[TableStatus,KeySchema[0].AttributeName]' \
    --output text

ACTIVE    LockID

# READ-ONLY: confirm the migrated state matches reality.
terraform plan -no-color -detailed-exitcode
echo "exit=$?"

No changes. Your infrastructure matches the configuration.
exit=0

# CONFIGURATION: exercise the lock end to end. Locking stays
# on; the point of the check is that the lock is acquired and
# released. Never add -lock=false here.
terraform apply -auto-approve

If any of the four checks fails, the backend migration is not complete. The old backend stays in place and the team investigates before retrying.