Infrastructure use cases — comparing IaC branches side by side; CI in a worktree; long-running checkouts

The previous four lessons established what worktrees are and how they behave. This lesson covers why an infrastructure team uses them: the three production scenarios where a worktree is materially better than the alternatives (stash-and-switch, fresh clone, separate repository). Each scenario has a measurable benefit, and each one maps to a routine workflow in IaC, CI, or operations.

Use case 1: side-by-side IaC branch comparison

The scenario: an engineer is mid-edit on feature/iam-rotation when a teammate asks for a review of feature/vpc-refactor. The review needs a working copy — Terraform plan, Ansible syntax check, or just a directory to run git diff against. The engineer cannot switch the main worktree (it would lose the in-progress edits) and does not want to clone (it is a 2 GB repository).

The worktree solution:

# From the main worktree, add a review worktree
git worktree add ~/work/iac-review-vpc feature/vpc-refactor

# In the review worktree, run the IaC tooling
cd ~/work/iac-review-vpc
terraform plan -out=tfplan-review
git diff main..feature/vpc-refactor

# Return to the main worktree; the in-progress edits are untouched
cd ~/work/iac
git status
# On branch feature/iam-rotation
# Changes not staged for commit: ...

The benefit is the elimination of context-switch loss. Stash, switch, review, switch back, unstash is a five-step procedure that loses partial work in the unstash step (conflicts, dropped staged changes, lost untracked files). The worktree is a two-step procedure (add, work) that leaves both edits intact.

flowchart LR
    MAIN["main worktree - feature/iam-rotation in progress"]
    REVIEW["review worktree - feature/vpc-refactor"]
    MAIN -. parallel IaC comparison .-> REVIEW
    REVIEW -. terraform plan / git diff .-> OUTPUT["review output"]
    OUTPUT --> DECISION["merge, request changes, or comment"]

The same pattern applies to bisecting a Terraform bug across two branches, to comparing a candidate production branch against main before a release, and to running parallel terraform plans against two candidate configurations. In all cases, the worktree provides isolation without the cost of a second clone.

Use case 2: CI running in a worktree

The scenario: a CI runner builds the repository on every push. The default checkout in most CI systems is a fresh clone of the repository per build. For a large IaC repo, the clone cost is significant — every build re-downloads the full object history.

The worktree-based CI checkout keeps a long-lived main worktree on the runner and creates a per-build worktree off it:

# On the CI runner, one-time setup (in the runner's bootstrap script)
git clone --bare git@github.com:acme/iac.git /var/lib/runner/iac.git
git worktree add /var/lib/runner/iac-main main
cd /var/lib/runner/iac-main
git remote set-url origin git@github.com:acme/iac.git

# Per-build checkout (in the job script)
cd /var/lib/runner/iac-main
git fetch origin
git worktree add /var/lib/runner/build-$BUILD_ID $BUILD_BRANCH
cd /var/lib/runner/build-$BUILD_ID
# ... run IaC build, terraform plan, ansible-lint, etc. ...
git worktree remove /var/lib/runner/build-$BUILD_ID --force

The cost saving is the clone cost on every build. Each build adds a worktree (cheap — local metadata plus working tree) and removes it at the end. The object database is fetched once per build (git fetch origin) and shared with the main worktree. For a repository with 2 GB of object history and 200 builds per day, the saving is approximately 400 GB of network transfer per day and 400 GB of disk write per day — both eliminated.

Use case 3: long-running checkouts for incidents and migrations

The scenario: an operator is managing a slow Terraform rollout that takes 36 hours to apply. The operator needs a working directory pinned to the exact commit that was applied, kept for the duration of the rollout, used to re-run terraform plan or inspect state at any point during the rollout.

The worktree solution:

# At the start of the rollout, pin a worktree to the exact commit
ROLLOUT_COMMIT=8a3f9d2
git worktree add --detach ~/work/iac-rollout-2026-08-21 $ROLLOUT_COMMIT

# Lock the worktree to prevent accidental removal during the rollout
cd ~/work/iac-rollout-2026-08-21
git worktree lock --reason "Terraform rollout in progress, see RUNBOOK-4711"

# Over the next 36 hours, use the directory for plan/state inspection
terraform plan -detailed-exitcode
terraform show
git log -1

# At the end of the rollout, unlock and remove
git worktree unlock ~/work/iac-rollout-2026-08-21
git worktree remove ~/work/iac-rollout-2026-08-21

The benefit is a directory that is guaranteed to be at the applied commit, for as long as the rollout runs, without any ongoing effort to keep it in sync. The lock (covered in lesson 06) prevents the worktree from being accidentally pruned during the rollout.

The same pattern applies to incident response (a worktree pinned to the commit that was running when the incident started, used for state inspection and rollback planning) and to migrations (a worktree pinned to the migration’s base commit, used to verify the migration’s pre-conditions as it runs).

Quantifying the savings

For a large infrastructure repository, the savings of worktree- based workflows over clone-based workflows are concrete:

For a CI fleet running 200 builds per day against a 2 GB repo, the daily saving is approximately 400 GB of network transfer and 400 GB of disk write — roughly $5–20/day in cloud egress and disk wear, and 30–60 minutes of cumulative build time per day that would otherwise be spent cloning.

Production discipline

1. Use worktrees for side-by-side IaC review, not stash-and- switch. The worktree is faster, lossless, and reviewable.
2. Configure CI runners with a long-lived main worktree plus per-build worktrees. The clone-cost saving compounds with every build.
3. Wrap CI build scripts in a trap that always removes the worktree, even on failure. Orphaned CI worktrees are the most common worktree-related incident.
4. Lock worktrees that back long-running operations (rollouts, incidents, migrations). The lock prevents accidental removal.
5. Remove worktrees when the operation ends. A worktree that outlives its purpose becomes a “branch already checked out” conflict waiting to happen.

Cross-course references

• Git, CI/CD & GitOps - Part II (GitModel) lesson 01 covers the working-tree/index/repository model that makes worktree reuse possible.
• CI/CD Pipeline Patterns - Part III (CheckoutStrategies) covers worktree-based CI runners in detail, including the trap-based cleanup pattern.
• Terraform for Production Sysadmins - Part XV (PlanApply) covers the long-running apply scenarios that drive the long-running-checkout use case.

Quiz