What infrastructure GitOps is — Terraform, Pulumi, and reconciliation

Kubernetes GitOps places the reconciler inside the cluster it controls. Argo CD, Flux, and the operator pattern assume the cluster is a long-running, network-reachable target that a controller can watch forever. Infrastructure GitOps cannot make that assumption. The cloud is not a pod; the cloud is a collection of accounts, regions, VPCs, and managed services spread across an API surface that no in-cluster controller can claim as its own. The reconciler has to live somewhere else, and the desired state has to live in Git while the observed state has to live in a state backend that Terraform or Pulumi owns.

flowchart LR
    A["Git repository"] -->|"PR opened"| B["Plan job"]
    B -->|"terraform plan"| C["State backend"]
    C -->|"current state"| B
    B -->|"comment"| D["Pull request"]
    D -->|"approved"| E["Apply job"]
    E -->|"terraform apply"| C
    C -->|"diff"| F["Cloud / infrastructure"]

The state backend is what makes infrastructure GitOps distinct. Kubernetes controllers read the cluster API server for observed state. Terraform and Pulumi read a state file or a cloud provider API for observed state. The state file is not a passive record; it is the cache that makes the plan deterministic. Losing it is not equivalent to losing an audit log; it is equivalent to losing the reconciler’s memory.

Where the controller lives

In Kubernetes GitOps, the controller runs as a deployment inside the cluster. In infrastructure GitOps, the controller runs in one of three places:

• A CI runner. Atlantis is the canonical example: a server that listens for pull-request webhooks, runs terraform plan, comments the diff on the PR, and runs terraform apply when a reviewer types the approval comment.
• An out-of-cluster operator. A long-running process in a dedicated VM, ECS task, or Kubernetes-on-Kubernetes control plane that watches Git and drives the cloud provider API.
• An in-cluster operator for in-cluster infra. The argoproj-labs/terraform-controller and the Pulumi Kubernetes Operator run inside a Kubernetes cluster, but they reconcile external infrastructure through cloud APIs. The cluster is the runner; the cloud is the target.

The Kubernetes-native model is a special case. The general case has no in-cluster reconciler and no shared API server. The state file is the API server; the cloud provider is the API server; the GitOps discipline is the same, but the components are different.

Declarative desired state versus authoritative state

The same word “state” means two different things. In HCL it means “the desired shape of the world”: a main.tf file that declares a VPC, three subnets, and a routing table. In Terraform state it means “the observed shape of the world as Terraform last saw it”: the JSON file that records the resource IDs, the attributes, and the dependencies Terraform must reconcile against.

GitOps cares about both. The first lives in Git under branch protection and signed commits; the second lives in a backend (S3 with state locking, Terraform Cloud, Consul) under access control and encryption. The two are reconciled by the plan job. A terraform plan reads the desired state from Git, reads the observed state from the backend, queries the cloud for what is actually there, and emits a diff.

sequenceDiagram
    participant G as Git
    participant P as Plan job
    participant S as State backend
    participant C as Cloud
    G->>P: read desired state at ref
    P->>S: read observed state
    P->>C: query live resources
    P->>P: diff desired vs observed vs live
    P->>G: comment diff on PR

A clean plan (no diff, no drift) is the steady state. A diff that the PR introduces is the change. A diff between observed state and the cloud is the drift. The plan surfaces all three at once, which is why the PR comment is the most important artefact in the loop.

The boundary between cluster and cloud

Kubernetes GitOps reconciles in-cluster resources: Deployments, Services, CRDs. Infrastructure GitOps reconciles cloud resources: VPCs, IAM policies, S3 buckets, DNS records. The boundary is not perfect. Some Kubernetes operators (Crossplane, AWS Controllers for Kubernetes, the Terraform controller) reconcile cloud resources from inside a cluster; some cloud-native tools (Cloud Custodian, Steampipe, Pulumi Cloud) reconcile Kubernetes resources from outside. The infrastructure GitOps part of this course treats the boundary as a deployment-time decision: where does the reconciler run, what is the trust relationship, and what is the blast radius of a compromised runner?

Production discipline

1. The state backend is part of the production blast radius. Treat it like a database: encryption at rest, access logs, point-in-time recovery where the backend supports it, and a separate identity from the one that runs plans.
2. Plan output is the contract. The PR comment with the diff is what reviewers approve. If the plan cannot be reproduced from the same commit, the contract is broken.
3. Apply is a separate identity from plan. A runner that can plan should not be able to apply; the approval step is the boundary. Atlantis enforces this by requiring a human-typed comment to unlock the apply.

Cross-course references

• This course, Part LXXII (GitOps Foundations) - the four principles applied here to a controller that lives outside the target system.
• Terraform for Production Sysadmins - Parts IX-XII (State) cover the state backend in depth; the plan/apply loop here is the GitOps version of the same loop.
• Kubernetes for Production Sysadmins - Parts on operators cover the reconcile pattern that argoproj-labs/terraform-controller and the Pulumi operator implement.

Quiz