Terraform and the Argo CD application controller — manifests as Git state

The argoproj-labs/terraform-controller extends Argo CD’s GitOps model to Terraform. Where Atlantis runs as an external service and reconciles per pull request, the terraform-controller runs inside the cluster as a Kubernetes operator and reconciles continuously, on a tick, against a custom resource that references a Terraform module. The cluster is the runner; the cloud is the target; the GitOps model is the same.

The model in one sentence: declare a Terraform module in Git, declare a Terraform custom resource in the cluster pointing at that module, and let the controller apply the module on every drift detection. The four OpenGitOps principles are satisfied: declarative (HCL), versioned (Git ref), pulled (the controller reads the ref), reconciled (the controller loops).

flowchart LR
    A["Git module ref"] --> B["Terraform CR"]
    B --> C["Controller"]
    C -->|"terraform init"| D["Module"]
    C -->|"plan"| E["Diff"]
    E -->|"approve"| F["Apply"]
    F -->|"write"| G["Output CR"]
    C -->|"state"| H["Workspace CR"]
    H -->|"lock"| I["Backend"]

The controller reads the Terraform resource, fetches the module at the declared ref, runs terraform init against a workspace declared by the Workspace custom resource, runs a plan, computes the diff, applies on approval or auto-approval, and writes the outputs to a Kubernetes secret or status field. The state file is held in the workspace, which can be a Kubernetes secret, an S3 bucket, or a remote backend; the controller does not care.

The Terraform and Workspace CRDs

The controller defines two custom resources. The Workspace holds configuration for a Terraform backend and a credential source; the Terraform holds a reference to a module, a workspace to use, and approval settings. The split matters because a single workspace can be reused across many Terraform resources - a shared state file with multiple modules writing to it - or one workspace per resource for isolation.

apiVersion: terraformcontroller.codes/v1alpha1
kind: Terraform
metadata:
  name: production-network
spec:
  sourceRef:
    module:
      source: git::https://github.com/acme/modules.git//network
      version: v1.4.2
  workspace: prod
  destroyResourcesOnDeletion: true
  approval: Auto

The sourceRef.module block is the GitOps declaration: a module address, a version (a Git ref or tag), and the path inside the repository. The controller pulls the module at the declared version, treats it as the desired state, and reconciles against the cloud. The approval field controls whether the apply needs an out-of-band annotation (Auto allows apply, Manual requires a human to set spec.approved=true).

The Workspace carries the backend configuration and the credentials:

apiVersion: terraformcontroller.codes/v1alpha1
kind: Workspace
metadata:
  name: prod
spec:
  module:
    source: git::https://github.com/acme/modules.git//network
  terraformVersion: 1.6.6
  values:
    region: us-east-1
  credentials:
    - name: aws-creds
      source: Secret
      secretRef:
        name: aws-credentials

The credentials block is where the boundary between cluster and cloud is enforced. The controller reads the secret at apply time, mounts it into the Terraform job, and the job uses it to talk to the cloud provider. The secret never leaves the cluster as plaintext, but it does leave the cluster as the temporary credentials in a Terraform process.

Auto-approval versus manual approval

The approval field is the production boundary. Auto produces a continuous-reconciler loop: the controller plans, detects drift or new commits, and applies. This is the GitOps-shaped answer - the controller is the reconciler - but it removes the human-in-the-loop approval that Atlantis provides by default.

Manual produces a controller-driven plan with a human-gated apply. The plan is recorded in the Terraform resource’s status, and the apply only runs after a human sets spec.approved=true. The boundary is the Kubernetes API; the audit trail is the API server log.

sequenceDiagram
    participant G as Git module
    participant T as Terraform CR
    participant C as Controller
    participant W as Workspace
    participant K as Cloud
    G->>C: new ref detected
    C->>W: acquire workspace
    C->>T: plan result in status
    alt approval = Manual
        T->>C: spec.approved=true
    end
    C->>K: terraform apply
    C->>T: outputs in status

The choice between Auto and Manual is not a configuration detail; it is a policy decision. Auto is appropriate for low-risk modules (DNS records, log buckets, S3 lifecycle policies). Manual is required for high-risk modules (IAM, network rules, anything that changes authentication or authorization).

Where secrets come from

Secrets in the controller model come from Kubernetes secrets, referenced by the Workspace’s credentials block. The controller does not encrypt them; it relies on etcd encryption at rest and RBAC on the secret resource. The boundary is the same boundary as the rest of Kubernetes: the secret is encrypted in etcd, decrypted by the API server, and mounted into the Terraform job as an environment variable or a file.

This is the operational difference from Atlantis. Atlantis fetches secrets from the CI runner’s environment or a secret-manager lookup; the controller fetches from Kubernetes. The trade-off is consistency (Kubernetes secrets are uniform across all controllers) versus flexibility (Atlantis can integrate with Vault, AWS Secrets Manager, or any backend the CI runner can authenticate against).

Production discipline

1. Choose approval mode per module. Auto for low-risk (DNS, log retention, tagging). Manual for IAM, network rules, anything that changes authentication or authorization.
2. Pin the module ref. A version: main reference means the controller reconciles against whatever the module’s default branch points at, which can change without a commit in the controller’s repository. Pin to a tag or a commit SHA.
3. State backend per workspace. A shared state file across modules is a coordination point. The controller serialises plans per workspace; two modules writing to one state file are two modules that step on each other.

Cross-course references

• This course, Part LXXII (GitOps Foundations) - the four principles in their canonical Kubernetes form, which the controller extends to Terraform.
• Terraform for Production Sysadmins - Parts IX-XII (State) cover the state backend in depth; the controller model uses the same backends.
• Kubernetes for Production Sysadmins - Parts on CustomResourceDefinitions and operators cover the controller pattern that terraform-controller implements.

Quiz