Lab 3: Resolve a realistic Terraform merge conflict with a state file

Objective

By the end of this lab you will have produced a real three-way merge conflict on a Terraform resource — two branches both edited the same attribute — and resolved it by combining the changes rather than choosing one side wholesale. You will also have recognised that the conflict is the second of two problems in the scenario, and that the first problem is a teammate committing a terraform.tfplan binary file and a terraform.tfstate to the index. The lab resolves the conflict; the policy decision is the deliverable.

The point is not “how do you edit conflict markers”. The point is “what does it mean for the resolution to be correct when the file is a Terraform resource declaration whose semantics care about whether you dropped a tag, a region, or a lifecycle rule?”

Architecture

Two feature branches diverge from a common base, each edits the same Terraform resource in a different way, and each commits a file that should never have been in version control. The merge is a three-way merge against main.

gitGraph
    commit id: "BASE" tag: "v1.0.0"
    branch feature/widen-cidr
    checkout main
    branch feature/add-tags
    checkout feature/widen-cidr
    commit id: "W1"
    commit id: "W2"
    checkout feature/add-tags
    commit id: "T1"
    commit id: "T2"
    checkout main
    merge feature/widen-cidr id: "MAIN-1"
    merge feature/add-tags id: "CONFLICT"

Both branches edit the same attribute (tags) on the same resource (aws_security_group.web). Both branches also add a file to the index that is generated output, not source: terraform.tfplan and terraform.tfstate. The merge is the first place anyone notices either mistake.

Requirements

Git 2.55.x on Linux or macOS.
Terraform 1.9.x (read-only use; the binary is installed but not invoked — the lab uses files that look like its output to simulate the conflict).
A clean working directory. Nothing outside $HOME/tf-merge-lab is touched.
No network access. No cloud credentials. No real AWS resources.

Scenario

Two engineers on the same Terraform repository opened competing pull requests. One widened the CIDR on aws_security_group.web and added a new ingress rule. The other added three tags (Owner, CostCenter, Environment) to the same security group. They both ran terraform plan locally and accidentally git add’d the terraform.tfplan and terraform.tfstate files that the CLI wrote. You are the reviewer merging second; the first branch merged cleanly, and now the second one has a real conflict on the tags block.

The merge will succeed mechanically. The conflict markers are not a stop sign — they are a question: which combination of the two edits is correct? Picking one side wholesale drops the other engineer’s work; combining them blindly may produce a syntactically valid but semantically broken result. The discipline of the lab is choosing correctly.

Tasks

Task 1 — Build the base repository

LAB="$HOME/tf-merge-lab"
rm -rf "$LAB"
mkdir -p "$LAB"
cd "$LAB"

git init -b main
git config user.email 'ops@example.com'
git config user.name  'Ops'

# The base commit: a single resource with no tags and a tight CIDR.
cat > main.tf <<'EOF'
terraform {
  required_version = ">= 1.9.0"
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }
}

resource "aws_security_group" "web" {
  name        = "web-tier"
  description = "ingress for the web tier"

  ingress {
    from_port   = 443
    to_port     = 443
    protocol    = "tcp"
    cidr_blocks = ["10.0.0.0/24"]
  }
}
EOF

git add main.tf
git commit -m 'BASE: initial web security group'

# Also commit a .gitignore that should have prevented this lab from
# being necessary. Its absence in the next two branches is the
# underlying mistake.
cat > .gitignore <<'EOF'
.terraform/
*.tfstate
*.tfstate.*
*.tfplan
crash.log
EOF

git add .gitignore
git commit -m 'BASE: add .gitignore for terraform artifacts'

git log --oneline

The base has one resource, no tags, and a CIDR of /24. Both branches will edit ingress.cidr_blocks (one widens it to /16, one leaves it alone) and tags (one adds three tags, one adds the same three plus an Owner override).

Task 2 — Branch A: widen the CIDR

# check-shell-blocks: allow-invalid
cd "$HOME/tf-merge-lab"

git branch feature/widen-cidr

git switch feature/widen-cidr

# W1: widen the CIDR, no tags yet
cat > main.tf <<'EOF'
terraform {
  required_version = ">= 1.9.0"
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }
}

resource "aws_security_group" "web" {
  name        = "web-tier"
  description = "ingress for the web tier"

  ingress {
    from_port   = 443
    to_port     = 443
    protocol    = "tcp"
    cidr_blocks = ["10.0.0.0/16"]
  }
}
EOF

git add main.tf
git commit -m 'W1: widen web tier CIDR to /16'

# W2: add three tags
cat > main.tf <<'EOF'
terraform {
  required_version = ">= 1.9.0"
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }
}

resource "aws_security_group" "web" {
  name        = "web-tier"
  description = "ingress for the web tier"

  ingress {
    from_port   = 443
    to_port     = 443
    protocol    = "tcp"
    cidr_blocks = ["10.0.0.0/16"]
  }

  tags = {
    Owner        = "platform"
    CostCenter   = "infrastructure"
    Environment = "production"
  }
}
EOF

git add main.tf
git commit -m 'W2: add three standard tags'

Branch A widened the CIDR and added three tags. The CIDR change and the tags block were committed in two separate commits so that the history reflects intent — the widen and the tagging are different operational decisions and should not be lumped together.

Task 3 — Branch B: add tags (the merge target)

cd "$HOME/tf-merge-lab"

git switch main

git branch feature/add-tags

git switch feature/add-tags

# T1: add the SAME three tags but with an Owner override — this is the
# collision with W2.
cat > main.tf <<'EOF'
terraform {
  required_version = ">= 1.9.0"
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }
}

resource "aws_security_group" "web" {
  name        = "web-tier"
  description = "ingress for the web tier"

  ingress {
    from_port   = 443
    to_port     = 443
    protocol    = "tcp"
    cidr_blocks = ["10.0.0.0/24"]
  }

  tags = {
    Owner        = "ops"
    CostCenter   = "infrastructure"
    Environment = "production"
  }
}
EOF

git add main.tf
git commit -m 'T1: tag web tier with ops owner'

# T2: add the generated plan and state files — the secondary mistake.
echo 'placeholder state content, not a real terraform state' \
  > terraform.tfstate
echo 'placeholder plan content, not a real terraform plan' \
  > terraform.tfplan

# Note: NO .gitignore would have caught this. The base commit HAS a
# .gitignore — branch B is branched from BEFORE the base .gitignore
# commit, which is the structural mistake we will reproduce.
git add terraform.tfstate terraform.tfplan
git commit -m 'T2: save plan and state for review'

Branch B is based on the BASE commit before the .gitignore was added. That is the structural mistake: branch B forked off main at the wrong point. In a real repository the same mistake is a branch that was opened before a colleague added .gitignore, or a branch that was rebased onto a base that does not include the policy file.

Task 4 — Merge A into main first

The first merge is clean because A and main only differ on main.tf, which has no conflicting edits.

cd "$HOME/tf-merge-lab"

git switch main
git merge --no-ff feature/widen-cidr \
  -m 'MAIN-1: widen web tier CIDR + standard tags'

git log --oneline --graph --decorate --all

This merge brings the widened CIDR and the three tags into main. Branch B still has its own three tags, with a different Owner value, and its own terraform.tfstate and terraform.tfplan files. Both differences are now live on top of main.

Task 5 — Merge B into main and trigger the conflict

cd "$HOME/tf-merge-lab"

git merge --no-ff feature/add-tags \
  -m 'CONFLICT: tag web tier with ops owner'
# expected output ends with:
# CONFLICT (content): Merge conflict in main.tf
# CONFLICT (modify/delete): ... terraform.tfplan ...
# Automatic merge failed; fix conflicts and then commit the result.

Git reports two conflicts. The content conflict on main.tf is the one the lab resolves. The modify/delete conflicts on the generated files are Git’s way of saying “one branch has the file and the other doesn’t” — because main does not have terraform.tfplan or terraform.tfstate, and branch B added both. Resolving those means delete them on branch B, because the generated files do not belong in the merge.

Task 6 — Inspect the conflict markers

# check-shell-blocks: allow-invalid
cd "$HOME/tf-merge-lab"

# Capture the conflicted file as it sits now, with markers.
cat main.tf > conflicted-input.txt

# Visualise the markers with line numbers.
grep -n -E '^<<<<<<<|^=======|^>>>>>>>' main.tf

The output identifies the lines that begin each side of the conflict and the separator between them. The <<<<<<< HEAD block is what main currently has (the merged A tags with Owner = platform). The >>>>>>> feature/add-tags block is what branch B wanted (the three tags with Owner = ops).

Task 7 — Resolve the content conflict manually

Open main.tf in your editor. Replace the entire conflict block — from the <<<<<<< line through and including the >>>>>>> line — with the correct combination.

# main.tf — post-resolution
terraform {
  required_version = ">= 1.9.0"
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }
}

resource "aws_security_group" "web" {
  name        = "web-tier"
  description = "ingress for the web tier"

  ingress {
    from_port   = 443
    to_port     = 443
    protocol    = "tcp"
    cidr_blocks = ["10.0.0.0/16"]
  }

  tags = {
    Owner        = "ops"
    CostCenter   = "infrastructure"
    Environment = "production"
  }
}

The resolution rules you applied:

CIDR kept at /16 — both branches’ BASE was /24, branch A widened to /16, branch B left at /24. The merge brought branch A into main first, so main is at /16. Branch B did not edit the CIDR, so its conflict marker on this attribute was empty. Use main.
Owner set to ops — branch B’s value was a deliberate override; use that.
CostCenter and Environment kept at the values main has from branch A — both branches agreed on these, no conflict.

Save the file as merge-result.tf and replace main.tf with it:

# check-shell-blocks: allow-invalid
cp merge-result.tf main.tf

# Sanity-check: no markers remain in the file.
grep -n -E '^<<<<<<<|^=======|^>>>>>>>' main.tf && {
  echo "still has conflict markers"
  exit 1
} || echo "markers removed"

# Sanity-check: the resource block parses as HCL.
terraform fmt -check main.tf && echo "fmt clean" || terraform fmt main.tf

The terraform fmt -check is optional but cheap: it confirms the file is syntactically valid HCL, which is the closest you can get to “the merge resolution is well-formed” without running a real plan.

Task 8 — Resolve the modify/delete conflicts by deleting

The right answer to a generated file that should not be in version control is to delete it. Both terraform.tfplan and terraform.tfstate need to be removed from the index on branch B.

cd "$HOME/tf-merge-lab"

# Remove the generated files from the working tree and from the
# index. Both: the file is on disk from the merge, and we do not
# want it tracked.
git rm --cached terraform.tfplan terraform.tfstate
rm -f terraform.tfplan terraform.tfstate

# Confirm no merge markers remain and no generated files are tracked.
git status
# expected: both files listed under "deleted by us" or similar, but
# the index is clean of them once git rm has run.

git rm --cached removes the file from the index without touching the working tree; the follow-up rm -f removes it from disk. The combination is the right answer for a generated file that should not be in the repository at all. For a generated file that should be ignored but might be regenerated, leave the working-tree copy alone and add it to .gitignore instead.

Task 9 — Commit the resolution and capture deliverables

# check-shell-blocks: allow-invalid
cd "$HOME/tf-merge-lab"

# Stage the resolved file and the deletions.
git add main.tf
git status
# expected: main.tf modified, terraform.tfplan deleted,
#           terraform.tfstate deleted.

# Use the merge commit message Git prepared; it already names both
# parents and the merge target.
git commit --no-edit

git log --oneline --graph --decorate --all

# Capture the deliverables.
cp main.tf merge-result.tf

{
  echo "# Files that should never have been committed"
  echo "# captured during the conflict, before resolution"
  echo
  git status --short
} > untracked.txt

# Final policy note — what the team should do next.
cat > policy-note.md <<'EOF'
# Policy note: Terraform state and plan files do not belong in git

The conflict in this merge surfaced a deeper mistake than the tag
collision on `aws_security_group.web`. Two engineers had committed
`terraform.tfstate` and `terraform.tfplan` to the index, on a
branch that forked before the team's `.gitignore` was added.

The right operational policy:

1. `terraform.tfstate` lives in a remote backend (S3, GCS, Terraform
   Cloud, etc.) with locking and encryption. It is never committed
   to the source repository.
2. `terraform.tfplan` is a build artefact of `terraform plan -out`.
   It is regenerated on every plan; committing it pins a stale plan
   in the history and conflates the reviewed change with the
   binary blob.
3. The `.gitignore` patterns for Terraform (`*.tfstate`,
   `*.tfstate.*`, `*.tfplan`, `.terraform/`) must be present in the
   very first commit of any new repository — not added later when
   the first accidental commit forces the conversation.
4. A pre-commit hook that runs `git status --porcelain | grep -E
   '\.tf(state|plan)$'` is the second line of defence.
EOF

The merge commit now points at the resolved main.tf, the two generated files are deleted, and the policy note documents the rule that should have prevented the conflict in the first place. The deliverables are the four files in $HOME/tf-merge-lab plus the git log output.

Validation

git log --oneline --graph --decorate --all shows the conflict merge commit with two parents, no <<<<<<< lines anywhere in the working tree, and the resolved main.tf as the post-merge state of main.
grep -E '^<<<<<<<|^=======|^>>>>>>>' main.tf returns nothing.
terraform fmt -check main.tf reports main.tf is formatted (run terraform fmt main.tf first if needed).
git ls-files | grep -E '\.tf(state|plan)$' returns nothing.
cat .gitignore includes *.tfstate, *.tfstate.*, *.tfplan, and .terraform/.
git log --all -- terraform.tfplan terraform.tfstate shows the history of the generated files. Branch B has two commits; main has none after the merge.
The deliverables merge-result.tf, conflicted-input.txt, untracked.txt, and policy-note.md exist and are non-empty.
policy-note.md mentions S3 or an equivalent remote backend, terraform plan -out, and the four .gitignore patterns.

Expected Outcome

A repository that has merged both branches, kept the wider CIDR, resolved the tag override in favour of branch B’s explicit choice, and removed the generated files. A separate document records what the team should have been doing all along.

$HOME/tf-merge-lab/
├── .git/
│   ├── MERGE_MSG                  # the merge commit message Git prepared
│   ├── objects/                   # all commits and trees from the lab
│   └── refs/heads/main            # the merge commit
├── .gitignore
├── conflicted-input.txt           # main.tf as it sat during the conflict
├── main.tf                        # post-resolution
├── merge-result.tf                # a copy of main.tf as a deliverable
├── policy-note.md                 # the policy the lab argues for
└── untracked.txt                  # status output from during the conflict

The repository is mergeable. The conflict resolution is correct. The policy that would have prevented the conflict is documented and delivered.

Troubleshooting

git merge reports a conflict but <<<<<<< does not appear in the file. Git produced a “conflict” because the modify/delete or mode-change check fired, but no text conflict exists. Look at git status for the file Git flagged; if it is one of the generated files, the resolution is git rm --cached as in Task 8, not a manual edit.

terraform fmt -check reports the file is unformatted. Run terraform fmt main.tf and re-stage. terraform fmt rewrites only whitespace and token spacing; it does not change semantics.

The resolved main.tf parses but the tags block looks wrong. A common mistake is to copy one side’s block and then leave the other side’s Owner line as an orphan tags.Owner = "ops" outside the block. The whole point of the resolution is that the file should be exactly the post-merge state with no leftover fragments. Read the file end-to-end after editing.

git log --all -- terraform.tfplan returns commits on both branches. That is expected — branch B introduced the file in T2, and the merge commit shows the deletion. What should be empty is git ls-files | grep tfplan, which proves the file is no longer in the index.

git commit --no-edit rejects with “Please tell me who you are”. The earlier git config user.email/user.name was on a different branch’s working tree and was lost when you switched. Re-run the two git config lines and retry.

Cleanup

The lab is entirely local. There is no remote, no cloud resource, no credential to revoke.

LAB="$HOME/tf-merge-lab"

# Keep the deliverables.
mv "$LAB"/merge-result.tf "$LAB"/policy-note.md \
   "$LAB"/untracked.txt "$LAB"/conflicted-input.txt \
   "$HOME"/ 2>/dev/null

rm -rf "$LAB"

find "$HOME" -maxdepth 1 -name 'tf-merge-lab' -print
# expected: (no output)

If you ran the lab inside an existing Terraform repository by accident, the merge and the policy note persist on it. Revert the merge with:

cd /path/to/that-repo
git revert -m 1 HEAD
git log --oneline --graph --decorate --all

The revert is a normal commit and the merge can be reapplied later once the team agrees on the policy note.

What You Learned

Conflict markers are a question, not a verdict. They tell you exactly what each side wants; the right resolution is the file you want to exist after the merge, which is a semantic judgement.
A modify/delete conflict on a generated file has only one right answer. Generated artefacts are policy violations independent of the merge; the resolution is to delete them.
Picking sides is a code smell. Choosing “ours” or “theirs” on a content conflict drops the other side’s work entirely. The correct resolution is almost always a combination.
The merge surfaced a deeper mistake. The state and plan files being in version control is the problem the team needs to fix; the tag collision is just where it became visible.
Policy is code. A .gitignore in the first commit of every new repository, a pre-commit hook that scans for generated files, and a backend configuration that puts state in S3 are the three controls that together prevent this entire class of conflict.
terraform fmt -check is the cheapest correctness signal. A merge resolution that does not parse is not a merge resolution; running fmt -check on the result is two seconds and one command.

Objectives

Prerequisites