Static checks and policy — the cheapest layer and what it catches

The base of the testing pyramid is the cheapest layer, and the cheapest layer is the one that pays for itself the fastest. Static checks and policy checks are the two layers that run without making any cloud call, without provisioning any container, and without opening any network connection. They are the layer every commit should pass before any other gate sees the change.

Static checks: syntactic

Static checks answer the question “is this file syntactically valid?”. They run the parser, the linter, or the schema validator against the file and report what does not match the expected shape. They are fast, deterministic, and offline.

The tools and what they cover:

• terraform fmt -check -recursive — formatting only. Reports files whose whitespace, indentation, or block alignment differs from terraform fmt’s canonical output. Exits non-zero if any file would be reformatted. The cheapest gate; milliseconds.
• terraform validate — schema and reference validation. Loads every .tf file, resolves references against the local state of variable definitions, locals, and module inputs, and reports unresolved references, type mismatches, and missing required arguments. Runs in seconds. Requires terraform init first, but the init is local-only and fast for a module without remote backends configured for the validate-only run.
• yamllint — YAML shape. Indentation, document markers, key duplication, truthy values. The first gate every YAML file passes.
• ansible-lint — Ansible shape. FQCN rules, task naming, module deprecations, role layout, idempotency hints. Runs in seconds.
• ansible-playbook --syntax-check — full Ansible parse. Resolves variables, modules, Jinja. Already covered in LI-04.
• kubeconform — Kubernetes manifest schema. Validates every manifest against the OpenAPI schema for the named Kubernetes version. Reports unknown fields, missing required fields, wrong types. Runs in milliseconds per manifest.
• tflint — Terraform-specific lint. Reports deprecated syntax, unused declarations, invalid resource attributes, AWS/GCP/Azure-specific mistakes that terraform validate cannot catch (wrong region format, wrong ARN shape, wrong instance type for the region).

None of these tools executes the configuration. None of them makes a cloud call. None of them even reads the cloud provider’s API. They read the file and apply a static rule set. The cost is in the milliseconds-to-seconds range; the savings come from short-circuiting every more expensive layer when a file fails.

Policy: semantic

Policy answers the question “does this file conform to the team’s rules?”. Syntactically valid files can still violate policy: an S3 bucket that parses cleanly and has no encryption, a security group that parses cleanly and is open to 0.0.0.0/0, a container that parses cleanly and runs as root. Policy tools read the file, build an internal representation, and apply a rule set that expresses what the file is allowed to do.

The tools and what they cover:

• tfsec — Terraform security and compliance. Hundreds of rules across AWS, Azure, GCP. Reports S3 without encryption, IAM policies that allow *:*, security groups open to the world, RDS without encryption at rest, missing access logging.
• checkov — multi-tool policy. Same class of rules as tfsec, with a different rule corpus and a different output format. Often used in parallel with tfsec to catch the rules the other misses.
• conftest (OPA/Rego) — general-purpose policy. Tests any structured file (Terraform plan JSON, Kubernetes manifest, Ansible vars file) against Rego rules the team writes. The most flexible tool; also the one that requires the team to write its own rules.
• kyverno — Kubernetes-native policy. Policies are written as YAML and applied to manifests before they reach the cluster. The right choice for Kubernetes-only policy that should also run as admission control in the cluster.

The policy layer is slightly slower than the static layer because it does more work, but it is still in the seconds range and still makes no cloud calls. It is the layer that catches “syntactically valid, semantically wrong.”

flowchart LR
    A[".tf / playbook.yml / manifest.yaml"] --> B[Static parse]
    B --> C{Valid?}
    C -->|no| D[Report - file:line]
    C -->|yes| E[Policy rules]
    E --> F{Conformant?}
    F -->|no| G[Report - rule ID]
    F -->|yes| H[Continue pipeline]

What the static and policy layers catch

The two layers, together, catch a large fraction of the mistakes that would otherwise reach the expensive layers:

• Formatting drift — terraform fmt -check, yamllint.
• Schema and reference errors — terraform validate, ansible-playbook --syntax-check, kubeconform.
• Deprecated or removed attributes — tflint, kubeconform with the right Kubernetes version.
• Missing required fields — kubeconform, ansible-lint.
• Unused declarations — tflint.
• FQCN and module naming — ansible-lint.
• Plain-text secrets in configuration — tflint, tfsec, checkov (best-effort, not a substitute for secret scanning).
• Security group ingress from 0.0.0.0/0 — tfsec, checkov.
• S3 without encryption — tfsec, checkov.
• Container running as root — tfsec, checkov, conftest, kyverno.
• Resource without a tag — tfsec, checkov, conftest.

The list is not exhaustive. The point is that every entry on the list is caught without a cloud call, without a container, without a network connection. The cost of catching them at the static and policy layers is the cost of running the tools.

What they do not catch

The two layers, together, miss a class of mistakes that requires asking the cloud:

• A policy that parses cleanly but does not grant the intended permission when evaluated by the cloud’s policy simulator.
• A DNS record that is configured correctly but does not resolve because the zone is wrong.
• A module output that has the right name but the wrong type for the downstream consumer.
• A network ACL that is configured to allow the right traffic but blocks it because of an unrelated deny rule earlier in the rule list.
• Anything that requires asking a running system.

The static and policy layers cannot catch these. The next layer — unit and module tests — catches some of them by mocking; the layer after that — disposable integration — catches all of them by asking the cloud.

Production discipline

1. terraform fmt -check -recursive is the first gate, always. Formatting drift that reaches any other gate is a wasted second.
2. Static and policy failures are hard gates, not warnings. The exit code must block the merge.
3. Policy rules live in version control alongside the configuration they govern. Policy-as-code is the rule; ad-hoc policy is the exception.
4. The policy corpus is reviewed periodically for false positives. A rule that fires on every PR is a rule that gets disabled or replaced.
5. The static and policy layers run on every PR. No exceptions, no bypasses, no “we’ll fix it next PR”.

Cross-course references

• This course, Part L (TerraformCI) - lessons git-cicd-gitops-l-02-fmt-and-validate and git-cicd-gitops-l-04-tflint-and-checkov are the Terraform-specific instantiations.
• This course, Part LI (AnsibleCI) - lessons git-cicd-gitops-li-02-yaml-and-playbook-linting and git-cicd-gitops-li-03-ansible-lint-and-fqcn-rules are the Ansible instantiations.
• This course, Part LII (KubernetesCI) - lessons git-cicd-gitops-lii-02-manifest-validation-kubeconform and git-cicd-gitops-lii-04-policy-conftest-and-kyverno are the Kubernetes instantiations.
• Terraform for Production Sysadmins - Part XXIV (ComplianceAsCode) covers how policy rules are versioned and reviewed.

Quiz