Terraform Architecture · Terraform

Terraform is not a single program. It is a small binary that loads plugins, writes a state file, and asks providers to do work. Every production surprise in Terraform — drift, partial apply, state corruption, surprising plan output — comes from misunderstanding one of these components.

The five components

+------------------------+        +------------------------+
| Configuration          |        | Providers              |
| (.tf files)            |        | (plugins, downloaded   |
|                        |        |  by `terraform init`)  |
| Declares desired state |        |                        |
+-----------+------------+        | Talk to real APIs      |
            |                     +-----------+------------+
            |                                 |
            v                                 v
        +---+---------------------------------+---+
        |                                         |
        |            Terraform Core              |
        |                                         |
        |  Reads configuration                    |
        |  Reads state                             |
        |  Builds the resource graph              |
        |  Calculates the diff                    |
        |  Asks providers to refresh              |
        |  Asks providers to apply the diff       |
        |                                         |
        +---------------------+-------------------+
                              |
                              v
        +---------------------+-------------------+
        |                                         |
        |            State                        |
        |  (terraform.tfstate, local or remote)    |
        |                                         |
        |  Records what Terraform believes exists |
        |                                         |
        +-----------------------------------------+

Configuration

The .tf files in a working directory. Declares resources, data sources, variables, outputs, modules, locals, provider configuration, state backend, and the required_version pin.

Configuration is the only thing that lives in source control. It is the source of truth for what Terraform believes the desired state is.

Providers

Plugins that talk to real APIs. Terraform ships with no providers bundled; providers are downloaded by terraform init from the public registry or from internal mirrors. A provider for AWS contains the API calls and the schema for aws_instance, aws_s3_bucket, and so on.

A provider is a stand-alone process. Terraform communicates with it over an internal RPC protocol. The provider is what owns the provider-specific knowledge of which attribute changes are replacement, which are in-place, and which require a separate API call.

Core

The downloaded terraform binary. It loads the configuration, calls the providers to refresh state, builds the resource graph, calculates the diff, and emits the plan. It does not know how to create an aws_s3_bucket; the core hands that work to the AWS provider.

State

A JSON file that records what Terraform currently believes the real-world infrastructure looks like. The state file is the only artifact that lets Terraform know what is already there without asking every provider for every resource on every run.

Real infrastructure

Whatever the providers talk to. A cloud account, a Proxmox cluster, a GitHub organisation, a DNS provider. Terraform does not own the infrastructure; the providers API does.

The flow of `terraform apply`

$ terraform apply
   │
   ├─► Read configuration
   │     ↓
   │     .tf files in the working directory
   │
   ├─► Read state
   │     ↓
   │     terraform.tfstate (or remote backend)
   │
   ├─► Provider refresh
   │     ↓
   │     For each resource, ask the provider:
   │     "is the real-world resource still as I last saw it?"
   │
   ├─► Construct resource graph
   │     ↓
   │     Resources, with edges for every reference
   │     and every depends_on
   │
   ├─► Calculate diff
   │     ↓
   │     For each resource, compare
   │     desired state (configuration) to
   │     current state (refreshed state + real world)
   │     → produce an action per resource: create, update, destroy, replace, no-op
   │
   ├─► Display plan
   │     ↓
   │     The "Plan: 3 to add, 0 to change, 0 to destroy." output
   │
   ├─► Approval (interactive)
   │     ↓
   │     "Do you want to perform these actions?" → yes / no
   │
   ├─► Execute provider operations
   │     ↓
   │     For each action, send the appropriate API call
   │     through the corresponding provider
   │     Update state after each successful operation
   │
   └─► Save state

The actual flow is more complex (the core uses a graph walker for parallelism, and the apply can be interrupted at any point), but this is the mental model that matters.

What state actually contains

The state file is a JSON document. The top-level keys are:

{
  "version": 4,
  "terraform_version": "1.9.8",
  "serial": 17,
  "lineage": "a7c3f0e2-...",
  "outputs": {},
  "resources": [
    {
      "mode": "managed",
      "type": "aws_instance",
      "name": "web",
      "provider": "provider[\"registry.terraform.io/hashicorp/aws\"]",
      "instances": [
        {
          "schema_version": 1,
          "attributes": {
            "id": "i-0abc123def456789",
            "ami": "ami-0e1bed4f",
            "instance_type": "t3.medium",
            "tags": {"Name": "web-01"}
          }
        }
      ]
    }
  ]
}

The state does not contain the users .tf configuration. It contains only the attributes that the provider returned after the last refresh, plus enough metadata to locate the resource in the providers API on the next refresh.

A state can grow large. A 5,000-resource estate is a state file of several megabytes. The size matters when you read Section XXIX later in the course.

The resource graph

Terraform constructs a graph of every resource, every data source, every module, and every output. The edges are the dependencies. Most are implicit — if aws_instance.web references aws_subnet.public.id, the instance depends on the subnet.

The graph is the source of:

Parallelism. Terraform executes independent resources concurrently, up to -parallelism=N (default 10).
Ordering. A resource does not start until all its dependencies have completed.
Diff calculation. The graph is walked topologically, and the diff is computed for each node.

The graph is also what terraform graph prints (in DOT format, which can be rendered to a picture with Graphviz).

What the core does not do

It does not write your code. The HCL is read-only as far as the core is concerned.
It does not decide what to deploy. It depends on the user to write the configuration.
It does not know what the provider is doing. The providers API behaviour is opaque.
It does not validate the result. terraform apply succeeded does not mean the infrastructure is healthy. The course returns to this in Part LXXIX.

What the providers do

A provider is more than a thin wrapper around an API. It must:

Define the resource schema (what attributes, what types, what blocks).
Decide which changes require replacement vs in-place update.
Validate inputs against the APIs actual constraints.
Translate API errors into Terraforms error vocabulary.
Read and write the state JSON for that resource type.

A provider is written by someone, and that someones understanding of the API determines the quality of the apply. Part XCVIII of the course returns to the supply-chain risk this creates.

What the core asks the provider to do

The core has a small vocabulary of operations it sends to a provider:

PlanResourceChange — given the prior state and the desired state, what is the diff?
ApplyResourceChange — perform the change described in the plan.
ReadResource — refresh real-world state for one resource.
ImportResource — adopt an existing real-world resource.
MoveResourceState — rename or move a resource within state.

The provider implements these operations. The core does not know how an aws_instance is created; it only knows that the provider returned a successful ApplyResourceChange.

What an S3 bucket looks like to Terraform

When Terraform plans an aws_s3_bucket change, the conversation between the core and the AWS provider looks roughly like:

core → provider: "I want to change the buckets tags from {env=dev}
        to {env=prod}. The current state has the old tags. The
        desired state has the new tags. What is the plan?"

provider: "Tag changes are in-place. No replacement. Send the
           PutBucketTagging API call."

core → AWS API:  PutBucketTagging(bucket=..., tags={env=prod})
AWS API → core:  200 OK

The provider makes the decision about replacement. The core makes the decision about ordering. The state file records what happened. If you write a provider that incorrectly marks every change as replacement, the core will dutifully destroy and recreate every resource — and the user will see a plan that says “20 to destroy, 20 to add”, not “20 to change”.

What this means for you

A few operational implications of the architecture:

State is the only source of truth for what Terraform knows. If it is wrong, Terraform is wrong.
Provider behaviour is opaque. Read the providers documentation for replacement rules, not just attribute documentation.
The graph is built from references. A for_each over a list of objects creates a resource per element; the graph reflects that.
The core is single-threaded for the plan, parallel for the apply. A plan that says “50 to add” with no dependencies will execute 50 API calls concurrently, up to -parallelism.
The state is read on every command. A corrupt state prevents every operation except terraform state.

What comes next

The next lesson is the basic workflow: fmt, validate, plan, apply, destroy. Every later lesson assumes the workflow is understood.