Objective
By the end of this lab you will have authored the artefacts that
implement three-layer Kubernetes manifest validation: kubeconform
for upstream OpenAPI schema validation, conftest with OPA Rego
for organisation-specific policy, and Kyverno policies that run
at admission time in production. The lab also authors a
Helm-values and Kustomize-overlay validation that ensures
rendered manifests pass both kubeconform and conftest.
The point of this lab is not any single tool — the labs in
Lessons LII-02 and LII-04 covered kubeconform and conftest
respectively. The point is the layering: schema validation
(catches malformed manifests), policy validation (catches
manifests that pass schema but violate policy), and runtime
admission control (catches policy violations at deploy time as
a final gate).
Architecture
A three-layer validation: schema (kubeconform), policy (conftest), runtime (Kyverno). The first two run in CI on every PR; the third runs at admission time in the cluster.
flowchart LR
A["manifests\nyaml"] --> B["schema\nkubeconform"]
A --> C["policy\nconftest + Rego"]
B -- pass --> D["merge allowed"]
C -- pass --> D
B -- fail --> Z1["PR red"]
C -- fail --> Z1
D --> E["kubectl apply"]
E --> F["runtime\nkyverno admission"]
F -- pass --> G["resource created"]
F -- fail --> Z2["apply rejected"]
The CI layer (kubeconform + conftest) catches violations before merge; the runtime layer (Kyverno) catches violations at deploy time. The two layers are paired: the CI layer is the prevention, the runtime layer is the safety net for resources applied outside the pipeline.
Requirements
- Git 2.55.x on Linux or macOS.
- A GitHub repository with Kubernetes manifests under
manifests/, Helm values underhelm-values/, Kustomize overlays underkustomize/, and policies underpolicy/. - No cluster required for the CI jobs (kubeconform and conftest run locally); Kyverno policies are reviewed but not enforced in the lab.
Scenario
A platform team manages a fleet of Kubernetes clusters. Every
manifest that lands in the cluster must (1) conform to the
upstream OpenAPI schema, (2) satisfy the team’s organisation
policies (no :latest tags, required labels, no privileged
containers), and (3) be admitted by the cluster’s admission
controller (Kyverno). The CI pipeline runs the first two on
every PR; the third runs in production as the final gate.
The lab builds the workflow, the policies, and the layered validation.
Tasks
Task 1 — Build the sample manifests
# check-shell-blocks: allow-invalid
LAB="$HOME/manifest-validation-lab"
rm -rf "$LAB"
mkdir -p "$LAB"
cd "$LAB"
git init -b main
git config user.email 'ops@example.com'
git config user.name 'Ops'
mkdir -p manifests
# A valid deployment with the team's required labels and a
# pinned image tag.
cat > manifests/web.yaml <<'EOF'
apiVersion: apps/v1
kind: Deployment
metadata:
name: web
labels:
app: web
owner: platform-team
env: production
spec:
replicas: 2
selector:
matchLabels:
app: web
template:
metadata:
labels:
app: web
owner: platform-team
env: production
spec:
containers:
- name: web
image: nginx:1.27-alpine
ports:
- containerPort: 80
resources:
requests:
cpu: 100m
memory: 64Mi
limits:
cpu: 500m
memory: 256Mi
securityContext:
runAsNonRoot: true
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
EOF
# A deliberately invalid manifest: missing apiVersion. This
# file demonstrates kubeconform's failure mode.
cat > manifests/broken.yaml <<'EOF'
kind: ConfigMap
metadata:
name: broken
data:
key: value
EOF
# A manifest that violates policy: uses :latest tag and lacks
# the `owner` label. This file demonstrates conftest's failure
# mode.
cat > manifests/violates-policy.yaml <<'EOF'
apiVersion: apps/v1
kind: Deployment
metadata:
name: violates
labels:
app: violates
spec:
replicas: 1
selector:
matchLabels:
app: violates
template:
metadata:
labels:
app: violates
spec:
containers:
- name: violates
image: nginx:latest # violates no-latest-tag policy
ports:
- containerPort: 80
EOF
git add manifests/
git commit -m 'initial: sample manifests including a broken and a policy-violating'
The repository has three manifests: a valid one, a broken one
(missing apiVersion), and a policy-violating one
(uses :latest tag, lacks owner label). The next tasks
wire up the validators.
Task 2 — Author the OPA Rego policy
# check-shell-blocks: allow-invalid
cd "$HOME/manifest-validation-lab"
mkdir -p policy/conftest
cat > policy/conftest/policy.rego <<'EOF'
package kubernetes.validations
# ─────────────────────────────────────────────────────────────────
# Rule 1: no `latest` tag on container images.
# ─────────────────────────────────────────────────────────────────
deny[msg] {
input.kind == "Deployment"
container := input.spec.template.spec.containers[_]
endswith(container.image, ":latest")
msg := sprintf("container '%s' uses :latest tag; pin to a specific version", [container.name])
}
deny[msg] {
input.kind == "StatefulSet"
container := input.spec.template.spec.containers[_]
endswith(container.image, ":latest")
msg := sprintf("container '%s' uses :latest tag; pin to a specific version", [container.name])
}
# ─────────────────────────────────────────────────────────────────
# Rule 2: required labels.
# ─────────────────────────────────────────────────────────────────
required_labels := ["app", "owner", "env"]
deny[msg] {
input.kind == "Deployment"
label := required_labels[_]
not input.metadata.labels[label]
msg := sprintf("Deployment '%s' is missing required label '%s'", [input.metadata.name, label])
}
# ─────────────────────────────────────────────────────────────────
# Rule 3: no privileged containers.
# ─────────────────────────────────────────────────────────────────
deny[msg] {
input.kind == "Deployment"
container := input.spec.template.spec.containers[_]
container.securityContext.privileged == true
msg := sprintf("container '%s' is privileged; remove privileged: true", [container.name])
}
# ─────────────────────────────────────────────────────────────────
# Rule 4: resource limits must be set.
# ─────────────────────────────────────────────────────────────────
deny[msg] {
input.kind == "Deployment"
container := input.spec.template.spec.containers[_]
not container.resources.limits.cpu
msg := sprintf("container '%s' has no CPU limit", [container.name])
}
deny[msg] {
input.kind == "Deployment"
container := input.spec.template.spec.containers[_]
not container.resources.limits.memory
msg := sprintf("container '%s' has no memory limit", [container.name])
}
# ─────────────────────────────────────────────────────────────────
# Rule 5: replicas must be at least 2 for production Deployments.
# ─────────────────────────────────────────────────────────────────
deny[msg] {
input.kind == "Deployment"
input.metadata.labels.env == "production"
input.spec.replicas < 2
msg := sprintf("production Deployment '%s' has replicas=%d; minimum is 2", [input.metadata.name, input.spec.replicas])
}
EOF
cat > policy/conftest/policy_test.rego <<'EOF'
package kubernetes.validations
# ─────────────────────────────────────────────────────────────────
# Test: no-latest-tag
# ─────────────────────────────────────────────────────────────────
test_no_latest_tag_violation {
deny["container 'web' uses :latest tag; pin to a specific version"] with input as {
"kind": "Deployment",
"metadata": {"name": "web"},
"spec": {"template": {"spec": {"containers": [{"name": "web", "image": "nginx:latest"}]}}
}
}
test_no_latest_tag_pass {
count(deny) == 0 with input as {
"kind": "Deployment",
"metadata": {"name": "web"},
"spec": {"template": {"spec": {"containers": [{"name": "web", "image": "nginx:1.27-alpine"}]}}
}
}
# ─────────────────────────────────────────────────────────────────
# Test: required-labels
# ─────────────────────────────────────────────────────────────────
test_missing_owner_label {
deny["Deployment 'web' is missing required label 'owner'"] with input as {
"kind": "Deployment",
"metadata": {"name": "web", "labels": {"app": "web", "env": "production"},
"spec": {"template": {"spec": {"containers": [{"name": "web", "image": "nginx:1.27"}]}}
}
}
# ─────────────────────────────────────────────────────────────────
# Test: privileged-containers
# ─────────────────────────────────────────────────────────────────
test_privileged_container {
deny["container 'web' is privileged; remove privileged: true"] with input as {
"kind": "Deployment",
"metadata": {"name": "web"},
"spec": {"template": {"spec": {"containers": [{"name": "web", "image": "nginx:1.27", "securityContext": {"privileged": true}]}}
}
}
# ─────────────────────────────────────────────────────────────────
# Test: production-replicas
# ─────────────────────────────────────────────────────────────────
test_production_replicas_violation {
deny["production Deployment 'web' has replicas=1; minimum is 2"] with input as {
"kind": "Deployment",
"metadata": {"name": "web", "labels": {"app": "web", "owner": "team", "env": "production"},
"spec": {"replicas": 1, "template": {"spec": {"containers": [{"name": "web", "image": "nginx:1.27"}]}}
}
}
test_production_replicas_pass {
count(deny) == 0 with input as {
"kind": "Deployment",
"metadata": {"name": "web", "labels": {"app": "web", "owner": "team", "env": "production"},
"spec": {"replicas": 2, "template": {"spec": {"containers": [{"name": "web", "image": "nginx:1.27"}]}}
}
}
EOF
git add policy/conftest/
git commit -m 'policy: OPA Rego policies for conftest'
The Rego policy has five rules: no :latest tags, required
labels, no privileged containers, required resource limits,
and minimum replicas for production. Each rule has a unit test
in policy_test.rego that the CI workflow runs with
conftest verify.
Task 3 — Author the Kyverno policy
# check-shell-blocks: allow-invalid
cd "$HOME/manifest-validation-lab"
mkdir -p policy/kyverno
cat > policy/kyverno/require-labels.yaml <<'EOF'
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: require-labels
spec:
validationFailureAction: Enforce
background: true
rules:
- name: require-app-owner-env-labels
match:
resources:
kinds:
- Deployment
- StatefulSet
validate:
message: "All Deployments and StatefulSets must have app, owner, and env labels."
pattern:
metadata:
labels:
app: "?*"
owner: "?*"
env: "?*"
EOF
cat > policy/kyverno/disallow-latest-tag.yaml <<'EOF'
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: disallow-latest-tag
spec:
validationFailureAction: Enforce
background: true
rules:
- name: no-latest-image-tag
match:
resources:
kinds:
- Deployment
- StatefulSet
- DaemonSet
validate:
message: "Container images must not use the :latest tag. Pin to a specific version."
pattern:
spec:
template:
spec:
containers:
- image: "!*:latest"
EOF
git add policy/kyverno/
git commit -m 'policy: Kyverno ClusterPolicies for runtime admission'
The Kyverno policies mirror the conftest policies but run at
admission time. validationFailureAction: Enforce means the
admission controller rejects resources that violate the
policy. The two policies are the runtime safety net for
resources applied outside the pipeline (for example, an
operator running kubectl apply directly).
Task 4 — Author Helm-values and Kustomize-overlay configurations
# check-shell-blocks: allow-invalid
cd "$HOME/manifest-validation-lab"
mkdir -p helm-values kustomize/overlays/production
cat > helm-values/values.yaml <<'EOF'
# Sample Helm values. The values file is rendered through
# `helm template` and the rendered output is validated.
image:
repository: nginx
tag: 1.27-alpine
pullPolicy: IfNotPresent
replicaCount: 2
labels:
app: web
owner: platform-team
env: production
resources:
limits:
cpu: 500m
memory: 256Mi
requests:
cpu: 100m
memory: 64Mi
podSecurityContext:
runAsNonRoot: true
fsGroup: 1000
containerSecurityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
EOF
cat > kustomize/overlays/production/kustomization.yaml <<'EOF'
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: production
labels:
- includeSelectors: true
pairs:
app: web
owner: platform-team
env: production
resources:
- ../../base
EOF
mkdir -p kustomize/base
cat > kustomize/base/kustomization.yaml <<'EOF'
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- web.yaml
EOF
cp manifests/web.yaml kustomize/base/web.yaml
git add helm-values/ kustomize/
git commit -m 'helm+kustomize: values and overlay for rendered validation'
The Helm-values file pins the image to 1.27-alpine (not
:latest), sets replicas to 2, and includes the required
labels. The Kustomize overlay adds the production namespace and
the required labels via the labels: transformer.
The CI workflow renders the Helm chart and the Kustomize
overlay, then validates the rendered output with kubeconform
and conftest. This catches policy violations that exist only
in the rendered output, not in the source manifests.
Task 5 — Author the CI workflow
# check-shell-blocks: allow-invalid
cd "$HOME/manifest-validation-lab"
mkdir -p .github/workflows
cat > .github/workflows/manifest-validation.yml <<'EOF'
name: manifest validation
on:
pull_request:
branches: [main]
push:
branches: [main]
permissions:
contents: read
concurrency:
group: manifest-validation-${ github.ref }
cancel-in-progress: ${ github.ref != 'refs/heads/main' }
jobs:
# ─────────────────────────────────────────────────────────────────
# Job 1: kubeconform against raw manifests
# ─────────────────────────────────────────────────────────────────
kubeconform-raw:
name: kubeconform (raw)
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1
- name: install kubeconform
run: |
curl -fsSLo kubeconform.tar.gz \
https://github.com/yannh/kubeconform/releases/latest/download/kubeconform-linux-amd64.tar.gz
tar -xzf kubeconform.tar.gz
sudo mv kubeconform /usr/local/bin/
- name: validate manifests
run: |
# -strict: fail on unknown fields
# -summary: print counts
# -kubernetes-version: validate against target cluster version
kubeconform -strict -summary \
-kubernetes-version 1.29.x \
-schema-location default \
-schema-location 'https://raw.githubusercontent.com/datreeio/CRDs-catalog/main/{.Group}/{.ResourceKind}_{.ResourceAPIVersion}.json' \
manifests/
# ─────────────────────────────────────────────────────────────────
# Job 2: conftest against raw manifests + policy tests
# ─────────────────────────────────────────────────────────────────
conftest-raw:
name: conftest (raw)
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1
- name: install conftest
run: |
curl -fsSLo conftest.tar.gz \
https://github.com/open-policy-agent/conftest/releases/latest/download/conftest_0.55.0_Linux_x86_64.tar.gz
tar -xzf conftest.tar.gz
sudo mv conftest /usr/local/bin/
- name: run policy tests
run: |
# Verify the policy itself (runs policy_test.rego).
conftest verify --policy policy/conftest/
- name: test manifests against policy
run: |
conftest test \
--policy policy/conftest/policy.rego \
--namespace kubernetes.validations \
manifests/
# ─────────────────────────────────────────────────────────────────
# Job 3: kubeconform + conftest against rendered manifests
# ─────────────────────────────────────────────────────────────────
rendered:
name: rendered (helm + kustomize)
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1
- name: install tools
run: |
curl -fsSLo kubeconform.tar.gz \
https://github.com/yannh/kubeconform/releases/latest/download/kubeconform-linux-amd64.tar.gz
tar -xzf kubeconform.tar.gz
sudo mv kubeconform /usr/local/bin/
curl -fsSLo conftest.tar.gz \
https://github.com/open-policy-agent/conftest/releases/latest/download/conftest_0.55.0_Linux_x86_64.tar.gz
tar -xzf conftest.tar.gz
sudo mv conftest /usr/local/bin/
curl -fsSL https://get.helm.sh/helm-v3.14.0-linux-amd64.tar.gz \
| tar -xz -C /tmp
sudo mv /tmp/linux-amd64/helm /usr/local/bin/
curl -fsSL https://github.com/kubernetes-sigs/kustomize/releases/download/kustomize%40v5.3.0/kustomize_v5.3.0_linux_amd64.tar.gz \
| tar -xz
sudo mv kustomize /usr/local/bin/
- name: render Helm chart
run: |
# A minimal Helm chart for the demo. The chart lives
# in helm-values/ but uses a `helm template` invocation
# that pulls values from values.yaml.
mkdir -p chart/templates
cp manifests/web.yaml chart/templates/web.yaml
helm template web chart/ \
--values helm-values/values.yaml \
--namespace production \
> rendered/helm-rendered.yaml
- name: render Kustomize overlay
run: |
kustomize build kustomize/overlays/production \
> rendered/kustomize-rendered.yaml
- name: validate rendered manifests
run: |
kubeconform -strict -summary \
-kubernetes-version 1.29.x \
rendered/
conftest test \
--policy policy/conftest/policy.rego \
--namespace kubernetes.validations \
rendered/
# ─────────────────────────────────────────────────────────────────
# Job 4: validate Kyverno policy syntax (does not require a cluster)
# ─────────────────────────────────────────────────────────────────
kyverno-policy:
name: kyverno policy lint
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1
- name: install kyverno CLI
run: |
curl -fsSLo kyverno.tar.gz \
https://github.com/kyverno/kyverno/releases/latest/download/kyverno-cli_v1.10.0_linux_amd64.tar.gz
tar -xzf kyverno.tar.gz
sudo mv kyverno /usr/local/bin/
- name: validate policies
run: |
kyverno apply policy/kyverno/ --dry-run
EOF
git add .github/workflows/manifest-validation.yml
git commit -m 'ci: kubeconform, conftest, and rendered validation'
The workflow has four jobs that run in parallel:
kubeconform-raw, conftest-raw, rendered, and
kyverno-policy. The first three validate manifests against
the upstream schema, the team’s policy, and the rendered
output. The fourth validates the Kyverno policy syntax (the
Kyverno CLI can validate policies without a cluster).
Task 6 — Document the validation layers
# check-shell-blocks: allow-invalid
cd "$HOME/manifest-validation-lab"
cat > validation-layers.md <<'EOF'
# Validation layers: schema, policy, runtime
This document is the canonical record of the three layers of
Kubernetes manifest validation. The workflow is the
implementation; this document is the rationale. Engineers
should be able to answer "what does each layer catch?" by
reading this file.
## Layer 1: Schema (kubeconform)
`kubeconform` validates against the upstream Kubernetes OpenAPI
schema. It catches:
- Missing or wrong `apiVersion`, `kind`, `metadata`.
- Wrong field names (e.g., `replicas` instead of
`replicaCount`).
- Wrong field types (e.g., `replicas: "2"` instead of
`replicas: 2`).
- Unknown fields (with `-strict`).
A schema failure is a malformed manifest. The CI workflow's
`kubeconform-raw` and `rendered` jobs both run `kubeconform`.
## Layer 2: Policy (conftest + OPA Rego)
`conftest` validates against organisation-specific Rego
policies. It catches:
- `:latest` tags (no unpinned images).
- Missing required labels.
- Privileged containers.
- Missing resource limits.
- Production-specific rules (e.g., `replicas >= 2`).
A policy failure is a manifest that passes schema but
violates the team's standards. The CI workflow's
`conftest-raw` and `rendered` jobs both run `conftest`.
## Layer 3: Runtime (Kyverno)
Kyverno runs as a Kubernetes admission controller. It catches
the same policy violations as conftest, but at deploy time.
This is the safety net for resources applied outside the
pipeline (e.g., `kubectl apply` from an operator's terminal).
A runtime failure is a resource that should have been
prevented by CI but slipped through. The CI workflow validates
the Kyverno policy syntax (`kyverno apply --dry-run`) but does
not run the admission controller itself; that requires a
cluster.
## Layer interaction
The three layers are paired:
kubeconform → schema correctness conftest → policy correctness kyverno → runtime enforcement
A manifest that passes `kubeconform` may fail `conftest` (a
schema-correct manifest that violates policy). A manifest
that passes `conftest` may fail Kyverno (a policy-correct
manifest that the cluster rejects for some other reason — for
example, a missing CRD).
## When each layer matters
| Scenario | Schema | Policy | Runtime |
|----------|--------|--------|---------|
| PR check | kubeconform | conftest | — |
| Deploy | — | — | Kyverno |
| Audit (raw) | kubeconform | conftest | — |
| Audit (rendered) | kubeconform | conftest | — |
EOF
git add validation-layers.md
git commit -m 'docs: validation layers description'
The validation-layers document is what the team reads when they ask “do we need both conftest and Kyverno?”. The answer is yes: conftest is prevention (CI), Kyverno is enforcement (runtime). The two are paired; neither alone is sufficient.
Task 7 — Validate the YAML and Rego structure
# check-shell-blocks: allow-invalid
cd "$HOME/manifest-validation-lab"
# Workflow parses and has four jobs.
python3 -c "
import yaml
with open('.github/workflows/manifest-validation.yml') as f:
doc = yaml.safe_load(f)
jobs = doc['jobs']
print('jobs:', list(jobs.keys()))
print('rendered steps:', [s.get('name', '?') for s in jobs['rendered']['steps']])
"
# Rego policy parses (rough check: look for `package` and `deny`).
python3 -c "
import re
with open('policy/conftest/policy.rego') as f:
content = f.read()
print('packages:', re.findall(r'^package\s+(\S+)', content, re.MULTILINE))
print('deny rules:', len(re.findall(r'^deny\[', content, re.MULTILINE)))
"
# Kyverno policies parse as valid YAML.
python3 -c "
import yaml, glob
for path in glob.glob('policy/kyverno/*.yaml'):
with open(path) as f:
doc = yaml.safe_load(f)
print(path, '→', doc['kind'], doc['metadata']['name'])
"
Expected output (excerpt):
jobs: ['kubeconform-raw', 'conftest-raw', 'rendered', 'kyverno-policy']
rendered steps: ['install tools', 'render Helm chart',
'render Kustomize overlay', 'validate rendered manifests']
packages: ['kubernetes.validations']
deny rules: 5
policy/kyverno/require-labels.yaml → ClusterPolicy require-labels
policy/kyverno/disallow-latest-tag.yaml → ClusterPolicy disallow-latest-tag
The workflow has four jobs; the Rego policy has one package
and five deny rules; the Kyverno policies are two
ClusterPolicy resources.
Task 8 — Capture the deliverables
cd "$HOME/manifest-validation-lab"
cp .github/workflows/manifest-validation.yml "$HOME/manifest-validation.yml"
cp policy/conftest/policy.rego "$HOME/policy.rego"
cp policy/conftest/policy_test.rego "$HOME/policy_test.rego"
cp policy/kyverno/require-labels.yaml "$HOME/require-labels.yaml"
cp policy/kyverno/disallow-latest-tag.yaml "$HOME/disallow-latest-tag.yaml"
cp helm-values/values.yaml "$HOME/values.yaml"
cp kustomize/overlays/production/kustomization.yaml "$HOME/kustomization.yaml"
cp validation-layers.md "$HOME/validation-layers.md"
ls -l "$HOME"/manifest-validation.yml \
"$HOME"/policy.rego \
"$HOME"/policy_test.rego \
"$HOME"/require-labels.yaml \
"$HOME"/disallow-latest-tag.yaml \
"$HOME"/values.yaml \
"$HOME"/kustomization.yaml \
"$HOME"/validation-layers.md
The deliverables are the eight files in $HOME, plus the
repository at $HOME/manifest-validation-lab.
Validation
.github/workflows/manifest-validation.ymlparses as valid YAML and has four jobs:kubeconform-raw,conftest-raw,rendered,kyverno-policy.policy/conftest/policy.regoparses as valid Rego and has fivedenyrules.policy/conftest/policy_test.regohas at least one test per rule.policy/kyverno/*.yamlare validClusterPolicyresources withvalidationFailureAction: Enforce.helm-values/values.yamlandkustomize/overlays/production/kustomization.yamlparse as valid YAML and render to manifests that passkubeconformandconftest.- Every
uses:reference in the workflow is a pinned commit SHA.
Expected Outcome
A three-layer Kubernetes manifest validation: schema (CI), policy (CI), runtime (admission). The CI workflow runs the first two on every PR, plus a rendered validation that catches violations that exist only in Helm or Kustomize output.
$HOME/manifest-validation-lab/
├── .github/workflows/manifest-validation.yml # the workflow
├── policy/
│ ├── conftest/
│ │ ├── policy.rego # OPA policy
│ │ └── policy_test.rego # OPA tests
│ └── kyverno/
│ ├── require-labels.yaml # admission policy
│ └── disallow-latest-tag.yaml # admission policy
├── manifests/ # raw manifests
├── helm-values/values.yaml # Helm values
├── kustomize/overlays/production/ # Kustomize overlay
└── validation-layers.md # the rationale
The workflow is the implementation; the policies are the rules; the document is the rationale.
Troubleshooting
kubeconform fails on a CRD. Add a -schema-location
flag pointing at the CRD definitions. The lab uses the
datreeio CRD catalog; teams with internal CRDs should host
them on an internal URL.
conftest reports a violation that the team disagrees with.
Either fix the manifest or update the Rego policy. A policy
change must update policy_test.rego to add a test for the
new behaviour.
helm template fails to render the chart. The chart’s
template references a value that is not in values.yaml. Add
the value or remove the reference.
kustomize build reports an error. The overlay references
a resource that does not exist in the base. Verify the path
in resources:.
kyverno apply --dry-run reports a policy error. The
Kyverno policy has a syntax error or references a field that
does not exist. The kyverno CLI’s error message points to the
line; fix and re-run.
The rendered job catches a violation that the raw job does not. A Kustomize overlay or Helm value is overriding the source. The rendered job is the one that catches it; the fix is in the overlay or values, not in the source.
Cleanup
LAB="$HOME/manifest-validation-lab"
mv "$LAB"/validation-layers.md "$HOME"/ 2>/dev/null
mv "$LAB/.github/workflows/manifest-validation.yml" \
"$HOME/manifest-validation.yml" 2>/dev/null
mv "$LAB/policy/conftest/policy.rego" "$HOME/policy.rego" 2>/dev/null
mv "$LAB/policy/conftest/policy_test.rego" \
"$HOME/policy_test.rego" 2>/dev/null
mv "$LAB/policy/kyverno/require-labels.yaml" \
"$HOME/require-labels.yaml" 2>/dev/null
mv "$LAB/policy/kyverno/disallow-latest-tag.yaml" \
"$HOME/disallow-latest-tag.yaml" 2>/dev/null
mv "$LAB/helm-values/values.yaml" "$HOME/values.yaml" 2>/dev/null
mv "$LAB/kustomize/overlays/production/kustomization.yaml" \
"$HOME/kustomization.yaml" 2>/dev/null
rm -rf "$LAB"
find "$HOME" -maxdepth 1 -name 'manifest-validation-lab' -print
# expected: (no output)
If you applied manifests to a real cluster during the lab,
remove them with kubectl delete:
kubectl delete -f manifests/ --ignore-not-found
kubectl delete namespace production --ignore-not-found
What You Learned
- Three layers, three tools.
kubeconformfor schema,conftestfor policy, Kyverno for runtime. The three are not alternatives; they are layers. Each catches what the others miss. - The rendered job is the one that catches override violations. A Kustomize overlay or Helm value can produce a manifest that violates policy even if the source manifest passes. The rendered job renders and validates the final output.
- Conftest and Kyverno policies must agree. The lab
authors both with the same rules (no
:latest, required labels). A divergence is a bug; the team’s discipline: any policy change updates both. - Rego policies need tests. A Rego policy without
policy_test.regocan break silently. The CI workflow runsconftest verifyto execute the tests and fail on any failure. - Kyverno is cluster-scoped, conftest is CI-scoped. A Kyverno policy deployed to a cluster is enforced for every resource in the cluster; a conftest policy is enforced only for manifests in the CI pipeline. The two are paired.
- The
-schema-locationflag is required for CRDs.kubeconformonly knows the upstream Kubernetes API types by default; for CRDs, point at a CRD catalog (datreeio) or an internal URL. validation-layers.mdis the rationale. The document is what the team reads when they ask “do we need all three?”. The answer is yes, and the document explains why.