Encryption at Rest for State

A state file in an unencrypted S3 bucket is a state file readable by anyone with s3:GetObject. A state file in an SSE-KMS-encrypted bucket is a state file readable only by IAM principals who have both S3 access and KMS decrypt access. The encryption does not eliminate the threat of an authorised insider reading state — the KMS key still has to be usable by the apply role — but it raises the bar: a leaked S3 credential without the corresponding KMS access produces a useless blob.

Two layers of encryption

Production state should have both server-side and (where possible) client-side encryption. They are not redundant; they address different threats.

Server-side encryption (SSE-KMS). S3 encrypts the object on write and decrypts on read. The encryption key is held by AWS KMS. The key policy controls who can use it.

Operator → S3 PutObject → S3 encrypts with KMS → encrypted blob in S3
Operator → S3 GetObject → S3 decrypts with KMS → cleartext state

Threat addressed: a stolen S3 credential without KMS access cannot read the state. A snapshot of the bucket (e.g. an S3 inventory export) does not contain cleartext state.

Client-side encryption (OpenTofu 1.7+). OpenTofu encrypts the state on the operator’s machine before sending it to S3. S3 stores an already-encrypted blob. The decryption key is held by the operator (a passphrase, a key file, or an external KMS via PBKDF2 + envelope encryption).

Operator → OpenTofu encrypts with passphrase → S3 stores encrypted blob
Operator → OpenTofu decrypts with passphrase → cleartext state

Threat addressed: an attacker who exfiltrates the S3 bucket contents (via a misconfigured replication, a compromised IAM role, a backup snapshot) gets an encrypted blob they cannot decrypt without the operator’s key.

S3 SSE-KMS configuration

The S3 bucket is configured with a bucket policy that enforces SSE-KMS:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "DenyUnencryptedObjectUploads",
      "Effect": "Deny",
      "Principal": "*",
      "Action": "s3:PutObject",
      "Resource": "arn:aws:s3:::tfstate-production/*",
      "Condition": {
        "StringNotEquals": {
          "s3:x-amz-server-side-encryption": "aws:kms"
        }
      }
    },
    {
      "Sid": "DenyWrongKMSKey",
      "Effect": "Deny",
      "Principal": "*",
      "Action": "s3:PutObject",
      "Resource": "arn:aws:s3:::tfstate-production/*",
      "Condition": {
        "StringNotEquals": {
          "s3:x-amz-server-side-encryption-aws-kms-key-id": "arn:aws:kms:us-east-1:123456789012:key/<key-id>"
        }
      }
    }
  ]
}

Two deny statements: any upload without SSE-KMS is rejected, and any upload with the wrong KMS key is rejected. Only the production state key can be used to write the state.

The backend configuration references the key:

terraform {
  backend "s3" {
    bucket         = "tfstate-production"
    key            = "global/terraform.tfstate"
    region         = "us-east-1"
    dynamodb_table = "tfstate-locks-production"
    encrypt        = true
    kms_key_id     = "arn:aws:kms:us-east-1:123456789012:key/<key-id>"
  }
}

The encrypt = true and kms_key_id fields tell the S3 backend to send the SSE-KMS header on every upload.

OpenTofu client-side state encryption

OpenTofu 1.7+ supports client-side state encryption. The configuration is in the OpenTofu block:

terraform {
  backend "s3" {
    bucket         = "tfstate-production"
    key            = "global/terraform.tfstate"
    region         = "us-east-1"
    dynamodb_table = "tfstate-locks-production"
  }

  cloud {
    # ... or use OpenTofu Cloud with built-in encryption
  }

  encryption {
    key_provider "pbkdf2" "encryption_key" {
      passphrase = var.state_encryption_passphrase
    }

    method "aes_gcm" "method" {
      keys = [key_provider.pbkdf2.encryption_key]
    }

    state {
      method   = method.aes_gcm.method
    }

    plan {
      method   = method.aes_gcm.method
    }
  }
}

The passphrase is provided as a variable, typically from a secrets manager or environment variable at apply time. The encryption key is derived from the passphrase using PBKDF2; the state and plan files are encrypted with AES-GCM using the derived key.

The production pattern: store the passphrase in AWS Secrets Manager; the CI pipeline fetches it at apply time; the passphrase never lands in a config file or a variable file.

KMS key management

The encryption key is the trust anchor. The discipline:

1. Dedicated key per environment. A separate KMS key for production, staging, and development. A compromised key in one environment does not affect the others.

2. Key policy limits to the apply IAM role only. No wildcard principals. No * actions. The minimum that allows the apply.

3. Key rotation enabled. AWS KMS automatic key rotation rotates the key material annually without changing the key ID. Enable it; the rotation is transparent to the apply.

4. Key access logged. CloudTrail logs every KMS use. The apply IAM role’s key usage is auditable. Investigate anomalies.

5. Key administrators separate from key users. Two IAM roles: the key admin who manages the key policy (rare, security team) and the key user who uses the key for encryption (the apply IAM role).

{
  "Sid": "AllowApplyRoleToUseKey",
  "Effect": "Allow",
  "Principal": {
    "AWS": "arn:aws:iam::123456789012:role/terraform-apply-production"
  },
  "Action": [
    "kms:Decrypt",
    "kms:GenerateDataKey",
    "kms:DescribeKey"
  ],
  "Resource": "*"
}

Validation

READ-ONLY

# KEY_ID: the KMS key protecting the bucket, as reported by the
# get-bucket-encryption call below (KMSMasterKeyID).
KEY_ID=9f8b7c6d-5e4f-4a3b-8c2d-1e0f9a8b7c6d

# Confirm the bucket is encrypted
aws s3api get-bucket-encryption --bucket tfstate-production
# Expected: SSEAlgorithm: aws:kms, KMSMasterKeyID: <key-arn>

# Confirm versioning is on
aws s3api get-bucket-versioning --bucket tfstate-production
# Expected: Status: Enabled

# Confirm the key policy
aws kms get-key-policy --key-id "$KEY_ID" --policy-name default

# Confirm the state object uses the right key
aws s3api head-object --bucket tfstate-production --key global/terraform.tfstate \
  | jq '{SSEAlgorithm: .ServerSideEncryption, KMSKeyId: .SSEKMSKeyId}'

For OpenTofu client-side encryption, the validation is in the apply path: a successful pull and push confirms the passphrase is correct.

Production failure modes

Symptom: “Error: server-side encryption is not enabled for this bucket”. Cause: the bucket was created without encryption; the backend attempted to upload without SSE-KMS. Recovery: enable SSE-KMS on the bucket (existing objects are not auto-encrypted); re-upload state via a refresh-only apply.

Symptom: “Error: kms:Decrypt not authorised”. Cause: the apply IAM role lost KMS access. Recovery: re-grant the role; verify with aws sts get-caller-identity and a dry-run.

Symptom: OpenTofu client-side encryption fails with “incorrect passphrase”. Cause: the passphrase is wrong or was rotated without updating the CI secret. Recovery: rotate the passphrase in the secrets manager; re-run.

Symptom: KMS key rotation has not happened in two years. Cause: automatic rotation was not enabled when the key was created. Recovery: enable automatic rotation; existing state is re-encrypted on next read (S3 handles this transparently).

Symptom: state is encrypted but readable by anyone with S3 access. Cause: SSE-S3 (AES-256 with AWS-managed key) was enabled, not SSE-KMS. SSE-S3 encryption is real but the key is held by AWS, not by the team’s KMS. Recovery: switch to SSE-KMS with a customer-managed key; the key policy becomes the access control.

Recovery

If the KMS key is compromised:

1. Disable the key (do not delete; the key material is still needed for decrypts of existing state).
2. Create a new KMS key with a new key policy.
3. Re-encrypt the state: copy each state version to itself with the new SSE-KMS key (S3 CopyObject with the new key).
4. Update the backend configuration to reference the new key.
5. Verify with terraform plan.

If the OpenTofu client-side encryption passphrase is lost:

1. The state is unrecoverable from the encrypted backup.
2. If the real-world infrastructure still exists, re-import every resource into a fresh state.
3. Generate a new passphrase and store it in the secrets manager.

What comes next

The next lesson covers backend access control: the IAM patterns that enforce least privilege, and the per-team boundary that limits blast radius.

Verification

• You can configure SSE-KMS on an S3 state bucket with a deny-policy that rejects unencrypted uploads.
• You can configure OpenTofu client-side state encryption with a passphrase from a secrets manager.
• You can write a KMS key policy that grants the apply IAM role only the minimum required actions.
• You can describe the threat model: encryption-at-rest protects against data exposure, not against authorised reads.