Skip to main content
RunBook Academy

KubernetesCXXIII · NetworkPolicy TroubleshootingNetworkPolicy troubleshooting

Policy testing and CI gates — the prevention strategy

Advanced⏱ ~14 minkubectlkyvernoconftest

What you'll learn

  • Reason about NetworkPolicy as code
  • Build a CI pipeline that validates the policy
  • Test the policy in staging before production
  • Identify the production failure modes of policy testing

Prerequisites

Verified against Kubernetes 1.34.x · kubeadm 1.34.x · kubectl 1.34.x · etcd 3.6.x · CoreDNS 1.11.x · containerd 1.7.x / 2.x · 2026-08-16

Not yet marked complete on this device.

NetworkPolicy as code is the prevention strategy. The CI pipeline validates the policy; the staging cluster tests the policy; the production cluster enforces the policy. The discipline is the same scale-free.

The policy as code

The NetworkPolicy is a Kubernetes object. The object’s YAML is the source of truth. The YAML is committed to a git repo; the CI pipeline validates the YAML; the GitOps controller applies the YAML to the cluster.

flowchart LR
    A[Git repo] --> B[CI pipeline]
    B --> C[Policy validation]
    C -->|Pass| D[GitOps controller]
    C -->|Fail| E[CI failure]
    D --> F[Staging cluster]
    F --> G[Production cluster]

The policy as code is the cluster’s network posture.

The CI pipeline

The CI pipeline validates the policy:

# Validate the policy with conftest (OPA)
conftest test policy.yaml --policy policies/

# Validate the policy with kubeconform
kubeconform -strict -summary policy.yaml

# Validate the policy with kubectl --dry-run
kubectl apply -f policy.yaml --dry-run=server

# Validate the policy with kyverno
# (requires the policy to be applied to a test cluster)

The CI pipeline is the policy’s gate.

The staging cluster

The staging cluster is the policy’s rehearsal. The staging cluster is a representative of production: the same CNI, the same CSI, the same workload shape.

# Substitute your own values before running:
SOURCE_POD=web-5f9c7d8b6c-2xk9p
TARGET_IP=192.0.2.24
TARGET_PORT=8080
TARGET_SVC=billing
TARGET_NS=prod

# Apply the policy to staging
kubectl apply -f policy.yaml -n prod

# Test the connection
kubectl exec -it "$SOURCE_POD" -n prod -- curl -v "$TARGET_IP:$TARGET_PORT"

# Test the egress
kubectl exec -it "$SOURCE_POD" -n prod -- nslookup "$TARGET_SVC.$TARGET_NS.svc.cluster.local"

The staging cluster is the policy’s gym.

The production cluster

The production cluster is the policy’s enforcement. The GitOps controller applies the policy to production; the CNI enforces the policy.

# Substitute your own values before running:
SOURCE_POD=web-5f9c7d8b6c-2xk9p
TARGET_IP=192.0.2.24
TARGET_PORT=8080

# Apply the policy to production
kubectl apply -f policy.yaml -n prod

# Verify the policy is enforced
kubectl exec -it "$SOURCE_POD" -n prod -- curl -v "$TARGET_IP:$TARGET_PORT"

The production cluster is the policy’s destination.

The policy validation

The policy validation is the CI pipeline’s gate. The validation checks:

  • Schema validity. The YAML is valid Kubernetes.
  • Best practices. The policy uses podSelector and namespaceSelector correctly.
  • Default-deny. The policy is paired with a default-deny.
  • DNS allowance. The policy allows DNS to kube-system.
# conftest policy example
package main

deny[msg] {
    input.spec.policyTypes[_] == "Egress"
    not input.spec.egress[_].to[_].namespaceSelector.matchLabels["kubernetes.io/metadata.name"] == "kube-system"
    msg := "Egress policy must allow DNS to kube-system"
}

The policy validation is the cluster’s network posture.

Common failures

  • Policy not validated. The CI pipeline does not validate the policy.
  • Policy not tested in staging. The policy is applied to production without testing.
  • Policy not committed. The policy is applied manually without committing to git.

Production discipline

The policy as code is the cluster’s network hypothesis. The discipline is to validate the policy in CI, test the policy in staging, enforce the policy in production. The network is the cluster’s connectivity; the policies are the cluster’s network posture.

  • Commit the policy to git before applying it. A policy applied manually never passed the CI gate.

Quiz

Knowledge check · 4 questions

  1. Q1. What is the role of the CI pipeline in policy as code?

  2. Q2. A NetworkPolicy should be applied to production without testing in staging.

  3. Q3. An operator wants to apply a new NetworkPolicy to production. The policy is in a git repo. The CI pipeline is set up. What is the workflow?

    The cluster is a 1.34.x kubeadm install. The policy is in a git repo. The CI pipeline runs conftest and kubeconform. The staging cluster mirrors production. The production cluster is managed by ArgoCD.

  4. Q4. Name three checks the CI pipeline should run on a NetworkPolicy and explain what each one does.

Passing score: 75%. Answers are checked in this browser.