Skip to main content
RunBook Academy

KubernetesCXXIII · NetworkPolicy TroubleshootingNetworkPolicy troubleshooting

Default-allow vs default-deny — the cluster network's posture

Advanced⏱ ~15 minkubectl

What you'll learn

  • Reason about the cluster's default network posture
  • Distinguish default-allow from default-deny
  • Diagnose the NetworkPolicy's effect on the cluster's network
  • Identify the production failure modes of default-allow vs default-deny

Prerequisites

Verified against Kubernetes 1.34.x · kubeadm 1.34.x · kubectl 1.34.x · etcd 3.6.x · CoreDNS 1.11.x · containerd 1.7.x / 2.x · 2026-08-16

Not yet marked complete on this device.

Every Pod in a fresh cluster can reach every other Pod, in any namespace, and nothing records that it happened. A NetworkPolicy changes that, but only for the Pods its podSelector matches and only in the directions named in policyTypes — a default-deny that lists Ingress alone leaves egress wide open. Policies are additive and never subtract, so isolation comes from a deny existing at all rather than from any ordering between rules.

The default network posture

A Kubernetes cluster’s default network posture is allow-all: every Pod can talk to every other Pod, every Service can be reached from any Pod, and the network is open.

flowchart LR
    A[Pod A] -->|Allow| B[Pod B]
    A -->|Allow| C[Pod C]
    B -->|Allow| C

The default is allow-all because the API server does not enforce network policies at the network layer. The CNI plugin may enforce policies, but the default is allow-all unless a policy is in place.

The NetworkPolicy

A NetworkPolicy is a Kubernetes object that specifies the allowed ingress and egress traffic for a set of Pods. The policy is enforced by the CNI plugin.

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny
  namespace: prod
spec:
  podSelector: {}
  policyTypes:
  - Ingress
  - Egress

The policy above is a default-deny policy: it selects all Pods in the namespace (podSelector: {}) and denies all ingress and egress traffic.

The default-deny pattern

The default-deny pattern is the canonical production pattern:

flowchart TD
    A[Namespace] --> B[default-deny NetworkPolicy]
    B --> C[Allow ingress from frontend]
    B --> D[Allow egress to database]
    B --> E[Allow egress to DNS]
    B --> F[Allow ingress from monitoring]

The pattern is:

  1. Default-deny all ingress and egress.
  2. Allow explicit traffic for known services.

The default-deny is the cluster’s network posture.

The diagnostic

The canonical diagnostic:

# Substitute your own values before running:
POLICY=default-deny-all
NS=prod
SOURCE_POD=web-5f9c7d8b6c-2xk9p
TARGET_IP=192.0.2.24
TARGET_PORT=8080

# 1. Check the NetworkPolicy
kubectl get networkpolicy -A

# 2. Describe the policy
kubectl describe networkpolicy "$POLICY" -n "$NS"

# 3. Test the connection from a Pod
kubectl exec -it "$SOURCE_POD" -- curl -v "$TARGET_IP:$TARGET_PORT"

# 4. Check the CNI's enforcement
# (CNI-specific: Calico, Cilium, etc.)
kubectl get felixconfiguration -o yaml -n calico-system

The diagnostic is the NetworkPolicy and the CNI’s enforcement.

Common failures

  • Default-allow is the assumption. The cluster operator believes the NetworkPolicy is enforced, but the CNI is not enforcing it.
  • Default-deny is too restrictive. The NetworkPolicy blocks legitimate traffic.
  • Selector mismatch. The NetworkPolicy’s selector does not match the Pods.
  • Egress policy missing. The NetworkPolicy denies ingress but allows egress, or vice versa.
flowchart TD
    A[Network failure] --> B{NetworkPolicy denies?}
    B -->|Yes| C[Selector mismatch]
    B -->|No| D{Pod-level DNS?}
    D -->|No| E[DNS is blocked]
    D -->|Yes| F{Network reachability?}
    F -->|No| G[CNI failure]
    F -->|Yes| H[Application failure]

The remediation

The remediation depends on the cause:

# Substitute your own values before running:
POLICY=default-deny-all
NS=prod

# Option 1: Add a default-deny NetworkPolicy
kubectl apply -f default-deny.yaml

# Option 2: Add an explicit allow rule
kubectl apply -f allow-frontend-to-backend.yaml

# Option 3: Fix the selector
kubectl patch networkpolicy "$POLICY" -n "$NS" -p '{"spec":{"podSelector":{"matchLabels":{"app":"billing"}}}}'

# Option 4: Verify the CNI's enforcement
# (CNI-specific)

The remediation is the network policy.

Production discipline

The default network posture is the cluster’s hypothesis. The discipline is to walk the canonical flow extended with the network policy, identify the failure mode, apply the remediation. The network is the cluster’s connectivity; the remediation is the network policy.

  • Default-deny is the production pattern. The default is allow-all; the policy is deny-by-default.
  • Explicit allow rules. Every legitimate traffic must be explicitly allowed.
  • Verify the CNI’s enforcement. The CNI is the policy enforcement layer.

Quiz

Knowledge check · 4 questions

  1. Q1. What is the default network posture of a Kubernetes cluster?

  2. Q2. A default-deny NetworkPolicy is the canonical production pattern.

  3. Q3. A namespace has just been given a default-deny NetworkPolicy and most of it has stopped working. Restore it without giving up the deny posture.

    A default-deny policy selecting every Pod, with both Ingress and Egress in policyTypes, was applied to namespace prod at 11:05. By 11:09, 18 of 22 Pods are failing readiness and logging name resolution timeouts. The four still healthy are the ones that connect to hard-coded IP addresses.

  4. Q4. Name three components of the default-deny pattern and explain what each one does.

Passing score: 75%. Answers are checked in this browser.