Environment Directories with Separate Backends

The directory-per-environment layout is the safest default for a production Terraform estate. The lesson teaches the concrete shape of the layout, how the backends are configured, how the variable files are managed, and how the layout maps onto a CI/CD pipeline.

The layout

A production estate with three environments and shared modules:

infra/
├── modules/
│   ├── network/
│   │   ├── main.tf
│   │   ├── variables.tf
│   │   ├── outputs.tf
│   │   └── versions.tf
│   └── compute/
│       ├── main.tf
│       ├── variables.tf
│       ├── outputs.tf
│       └── versions.tf
├── envs/
│   ├── dev/
│   │   ├── main.tf
│   │   ├── backend.tf
│   │   ├── variables.tf
│   │   ├── dev.tfvars
│   │   ├── dev.tfbackend
│   │   └── versions.tf
│   ├── staging/
│   │   ├── main.tf
│   │   ├── backend.tf
│   │   ├── variables.tf
│   │   ├── staging.tfvars
│   │   ├── staging.tfbackend
│   │   └── versions.tf
│   └── prod/
│       ├── main.tf
│       ├── backend.tf
│       ├── variables.tf
│       ├── prod.tfvars
│       ├── prod.tfbackend
│       └── versions.tf
└── .gitignore

The modules/ directory is reusable code. The envs/<name>/ directories are roots. Each root is a separate Terraform working directory with its own state.

The .gitignore excludes runtime artefacts:

# .gitignore
.terraform/
.terraform.tfstate
.terraform.tfstate.*
crash.log
*.tfplan
override.tf
override.tf.json
*_override.tf
*_override.tf.json

The root module

Each envs/<name>/main.tf instantiates the shared modules with environment-specific inputs:

# envs/prod/main.tf
module "network" {
  source = "../../modules/network"

  cidr_block = var.cidr_block
  env_name   = var.env_name
}

module "compute" {
  source = "../../modules/compute"

  subnet_ids     = module.network.private_subnet_ids
  instance_type  = var.instance_type
  instance_count = var.instance_count
  env_name       = var.env_name
}

The variables are declared in variables.tf:

# envs/prod/variables.tf
variable "cidr_block" {
  type        = string
  description = "CIDR block for the production VPC"
}

variable "env_name" {
  type        = string
  description = "Environment name (prod, staging, dev)"
  default     = "prod"
}

variable "instance_type" {
  type        = string
  description = "EC2 instance type for production compute"
}

variable "instance_count" {
  type        = number
  description = "Number of production instances"
}

The variable values live in prod.tfvars:

# envs/prod/prod.tfvars
cidr_block     = "10.0.0.0/16"
instance_type  = "m5.xlarge"
instance_count = 3

The backend

Each environment has its own backend configuration. The recommended pattern is partial configuration, with the actual values supplied via a *.tfbackend file or -backend-config flags at init time:

# envs/prod/backend.tf
terraform {
  backend "s3" {
    bucket = "mycompany-terraform-state"
    key    = "prod/terraform.tfstate"
    region = "us-east-1"
  }
}

# envs/prod/prod.tfbackend
bucket         = "mycompany-terraform-state"
key            = "prod/terraform.tfstate"
region         = "us-east-1"
dynamodb_table = "terraform-locks-prod"
encrypt        = true
role_arn       = "arn:aws:iam::PROD_ACCOUNT:role/terraform-execution"

The init command uses partial configuration:

terraform init -backend-config=prod.tfbackend

The advantage of partial configuration is that the static backend.tf carries the canonical key, while the credentials and lock table are loaded from a file that is not committed.

The tfvars discipline

The variable files follow three rules:

1. One tfvars per environment. prod.tfvars, staging.tfvars, dev.tfvars. Each file is committed to Git.

2. No terraform.tfvars in production. A root-level terraform.tfvars would apply to every environment in the directory. Production must be explicit. Use -var-file on every command, or use a wrapper script.

3. Secrets do not live in tfvars. Passwords, API keys, and database credentials come from a secret manager (AWS Secrets Manager, HashiCorp Vault), not from a .tfvars file.

The third rule is the most important. A committed .tfvars file is a public record. A committed .tfvars file with a secret is a security incident waiting to happen.

# Bad: secret in tfvars
db_password = "supersecret"

# Good: secret from a data source
data "aws_secretsmanager_secret_version" "db" {
  secret_id = "prod/db/password"
}

resource "aws_db_instance" "primary" {
  password = jsondecode(data.aws_secretsmanager_secret_version.db.secret_string)["password"]
}

The CI/CD pipeline reads the secret from the secret manager at apply time. The configuration code references the secret by name; the value never lives in the repository.

A wrapper script

A common pattern is a wrapper script per environment that selects the right tfvars and tfbackend:

#!/usr/bin/env bash
# bin/apply-prod.sh
set -euo pipefail

ENV=prod
TFVARS="${ENV}.tfvars"
TFBACKEND="${ENV}.tfbackend"

cd "envs/${ENV}"

terraform init -backend-config="${TFBACKEND}"
terraform validate
terraform plan -var-file="${TFVARS}" -out="${ENV}.tfplan"

# Manual approval gate in CI
if [[ "${CI:-false}" == "true" ]]; then
  echo "Plan written. Awaiting manual approval."
  exit 0
fi

terraform apply "${ENV}.tfplan"

The wrapper is committed to Git. The CI/CD pipeline invokes the wrapper. The manual approval gate is in CI, not in the wrapper, so the wrapper can be tested locally.

Wiring into CI/CD

A typical CI/CD pipeline for the directory-per-env layout:

# .github/workflows/terraform.yml
name: Terraform
on:
  pull_request:
    paths:
      - 'infra/**'
  push:
    branches: [main]
    paths:
      - 'infra/**'

jobs:
  plan:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        env: [dev, staging, prod]
    steps:
      - uses: actions/checkout@v4
      - name: Configure AWS credentials
        uses: aws-actions/configure-aws-credentials@v4
        with:
          role-to-assume: ${{ vars[format('terraform_role_{0}', matrix.env)] }}
          aws-region: us-east-1
      - name: terraform init
        working-directory: infra/envs/${{ matrix.env }}
        run: terraform init -backend-config=${{ matrix.env }}.tfbackend
      - name: terraform plan
        working-directory: infra/envs/${{ matrix.env }}
        run: terraform plan -var-file=${{ matrix.env }}.tfvars -out=${{ matrix.env }}.tfplan
      - name: upload plan
        uses: actions/upload-artifact@v4
        with:
          name: tfplan-${{ matrix.env }}
          path: infra/envs/${{ matrix.env }}/${{ matrix.env }}.tfplan

  apply:
    needs: plan
    if: github.event_name == 'push' && github.ref == 'refs/heads/main'
    runs-on: ubuntu-latest
    strategy:
      matrix:
        env: [dev, staging, prod]
    steps:
      - uses: actions/checkout@v4
      - name: Configure AWS credentials
        uses: aws-actions/configure-aws-credentials@v4
        with:
          role-to-assume: ${{ vars[format('terraform_role_{0}', matrix.env)] }}
          aws-region: us-east-1
      - name: terraform init
        working-directory: infra/envs/${{ matrix.env }}
        run: terraform init -backend-config=${{ matrix.env }}.tfbackend
      - name: terraform apply
        working-directory: infra/envs/${{ matrix.env }}
        run: terraform apply ${{ matrix.env }}.tfplan

The matrix runs a plan per environment. The credentials are sourced from GitHub Actions variables; the secrets do not live in the workflow file. The apply only runs on push to main, and only after the plan is uploaded as an artefact.

Validation

The validation commands for a directory-per-env layout:

# Verify each environment has its own backend
for env in dev staging prod; do
  echo "=== $env ==="
  grep -h 'key' "envs/$env/backend.tf"
done

=== dev ===
    key    = "dev/terraform.tfstate"
=== staging ===
    key    = "staging/terraform.tfstate"
=== prod ===
    key    = "prod/terraform.tfstate"

A plan with no changes is the validation that nothing has drifted:

cd envs/prod
terraform plan -var-file=prod.tfvars

No changes. Your infrastructure matches the configuration.

A failed terraform validate is a signal that the configuration is broken before the plan:

cd envs/prod
terraform validate

What comes next

The next lesson is multi-account and multi-project production: the next boundary up from the directory, where the environment lives in a separate AWS account, GCP project, or Azure subscription.

Verification

• ls infra/envs/ lists one directory per environment.
• Each infra/envs/<name>/backend.tf references a different state key.
• grep -l 'password\s*=' infra/envs/*/*.tfvars returns no results: no secrets are committed in tfvars.
• terraform plan -var-file=prod.tfvars from infra/envs/prod/ shows prod resources; the same command from infra/envs/staging/ shows staging resources.
• The IAM role used to apply infra/envs/prod/ cannot read the state bucket used by infra/envs/staging/.