Skip to main content
RunBook Academy

Git, CI/CD & GitOpsCXI · Kubernetes Delivery PipelineCIRenderValidate

Manifest render and validate — the CI side of the pipeline

Advanced⏱ ~27 mingitkubectlhelmkustomize

What you'll learn

  • Run helm template and kustomize build to render manifests in CI
  • Apply kubeconform and conftest as schema and policy gates on rendered output
  • Distinguish what schema validation catches from what policy validation catches
  • Recognise why render-in-CI is the only place render should ever run before sync

Prerequisites

Verified against Git 2.55.x teaching target; 2.40+ minimum · GitHub Actions continuous service; Aug 2026 documentation baseline · Argo CD v3.5.x teaching target; v3.0+ minimum · Flux v2.9.x · Sigstore Cosign v3.1.x · SLSA v1.2 · OCI Distribution Specification v1.1 · Git LFS v3.7.1 · Kubernetes (cross-course target) 1.36.x

Not yet marked complete on this device.

The CI side of the Kubernetes pipeline exists to produce one artefact: a rendered, validated manifest bundle, keyed by commit and environment, that the GitOps controller can consume without re-derivation. The render step and the validation step are deliberately separate. Render turns a chart or a kustomize tree into plain YAML; validation checks that the plain YAML is well-formed and policy-compliant. Neither step touches the cluster.

Two render verbs

The two tools that produce a rendered manifest from a Kubernetes repository are helm template and kustomize build. They are not interchangeable: they answer different questions.

flowchart LR
    A["Source commit"] --> B{"Helm chart or Kustomize tree?"}
    B -->|"chart"| C["helm template ./chart"]
    B -->|"tree"| D["kustomize build overlays/prod"]
    C --> E["Rendered YAML"]
    D --> E
    E --> F["kubeconform -strict -kubernetes-version 1.30.0"]
    F --> G["conftest test --policy policies/"]
    G --> H["Rendered and validated bundle"]

helm template is the right verb when the source is a Helm chart and the values file is the per-environment input:

helm template ./chart \
  --values values-prod.yaml \
  --namespace prod \
  --include-crds

kustomize build is the right verb when the source is a base-plus-overlays tree and the patch files are the per-environment input:

kustomize build overlays/prod

Both produce plain YAML on standard output. That YAML is the artefact the rest of the pipeline consumes.

Two validation gates

A rendered manifest that does not parse against the Kubernetes API schema cannot be applied. A rendered manifest that parses but violates policy is worse: it applies cleanly and ships a violation. CI runs both gates against the rendered YAML.

The schema gate:

kubeconform -strict -summary \
  -kubernetes-version 1.30.0 \
  -summary \
  rendered/

The -strict flag fails on unknown fields. The -kubernetes-version flag pins the schema set so a CI job does not silently start validating against a newer API after a base image upgrade.

The policy gate:

conftest test --policy policies/ rendered/

Conftest evaluates each rendered file against Rego rules. Typical rules: no privileged containers, no host network, all images pinned by digest, every workload declaring resource limits, every namespace carrying a label that names its owner.

The two gates are deliberately narrow:

  • kubeconform catches malformed manifests and unknown fields. It does not know what your team considers a policy violation.
  • conftest catches policy violations. It does not know whether the YAML is syntactically valid against the API server.

A pipeline that runs only kubeconform ships policy violations; a pipeline that runs only conftest ships YAML the cluster will reject.

What render-and-validate does not do

Render and validate are CI’s job. They do not:

  • Touch the cluster. CI holds no kubeconfig scoped to production.
  • Decide whether the change should ship. That is human approval on the pull request.
  • Re-render at deploy time. The controller consumes the artefact that was rendered at PR time.
  • Reconcile. The controller does that on a loop after merge.

The boundaries are not stylistic. They are the credential and identity boundaries that make the pipeline auditable.

Production discipline

  1. Render with the same flags in CI and in packaging. If helm template runs with --include-crds in CI, the controller must consume output that was rendered the same way.
  2. Pin the schema set. kubeconform -kubernetes-version is a build argument, not a constant. Bump it deliberately when the cluster is upgraded.
  3. Policy lives next to manifests. Rego files in the same repo as the YAML they evaluate; PRs that change manifests update the policy in the same review.
  4. Render artefacts are immutable. Once stored, the rendered bundle for commit 8a3f9d2 never changes. If it needs to change, a new commit is made.
  5. Render failures are merge blockers. A kubeconform failure or a conftest failure must fail the CI job, not warn.

Cross-course references

  • Kubernetes for Production Sysadmins - Parts XI-XIV cover Argo CD’s source-side validation hooks that complement CI render.
  • Containers for Production Sysadmins - Parts XI-XIV cover digest-pinning, the most common policy rule in conftest.
  • This course, Part LII (KubernetesCI) - lessons git-cicd-gitops-lii-02 through git-cicd-gitops-lii-05 cover the validation gates in detail.
  • This course, Part LVII (Approvals) - covers the human gate that runs after CI validates the render.

Quiz

Knowledge check · 4 questions

  1. Q1. What is the role of `helm template` or `kustomize build` in the CI side of the Kubernetes pipeline?

  2. Q2. Running kubeconform alone is not sufficient for production validation because Kubernetes will reject any malformed manifest on apply.

  3. Q3. Why is the rendered manifest bundle stored as an immutable artefact in CI?

  4. Q4. Diagnose a pipeline that skips conftest and ships a policy violation, and prescribe the correction.

    A team runs helm template and kubeconform in CI, then merges to the GitOps branch. Argo CD syncs the manifests and the rollout completes. Two weeks later, a security review discovers that a recent PR introduced a Deployment with `hostNetwork: true` and no resource limits. kubeconform never flagged it because the YAML parsed; conftest was not part of the pipeline. The cluster has been running the violation in production for two weeks, and the audit trail shows the controller applied it under its own service account.

Passing score: 75%. Answers are checked in this browser.