Egress and DNS rules — the outbound traffic path

Egress rules and DNS are the outbound traffic path. The default-deny policies must allow DNS to kube-system. The diagnostic is the policy and the DNS resolution. The discipline is the same canonical flow extended with the egress rules.

The egress rules

A NetworkPolicy’s spec.egress rules specify the allowed outbound traffic. The default is to deny all egress if the policy includes policyTypes: [Egress].

flowchart TD
    A[Pod] --> B{egress policy?}
    B -->|Default-deny| C[Denied]
    B -->|Allow DNS| D[DNS allowed]
    B -->|Allow DB| E[Database allowed]

The egress rules are the cluster’s outbound policy.

The DNS allowance

A default-deny egress policy must allow DNS to the kube-system namespace. The DNS allowance is the most common oversight.

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-dns
  namespace: prod
spec:
  podSelector: {}
  policyTypes:
  - Egress
  egress:
  - to:
    - namespaceSelector:
        matchLabels:
          kubernetes.io/metadata.name: kube-system
      podSelector:
        matchLabels:
          k8s-app: kube-dns
    ports:
    - protocol: UDP
      port: 53
    - protocol: TCP
      port: 53

The DNS allowance is the cluster’s DNS resolution.

The diagnostic

The canonical diagnostic:

# Substitute your own values before running:
NS=prod
SOURCE_POD=frontend-5f9c7d8b6c-2xk9p
TARGET_IP=192.0.2.40      # ClusterIP of the Service being reached
TARGET_PORT=8080

# 1. Check the NetworkPolicy's egress rules
kubectl get networkpolicy -n "$NS" -o yaml

# 2. Test the DNS resolution
kubectl exec -it "$SOURCE_POD" -- nslookup billing.prod.svc.cluster.local

# 3. Test the egress to a specific target
kubectl exec -it "$SOURCE_POD" -- curl -v "$TARGET_IP:$TARGET_PORT"

# 4. Check the CNI's enforcement
# (CNI-specific)

The diagnostic is the policy’s egress rules and the connection attempts.

Common failures

• DNS blocked. The policy denies egress to the kube-system namespace. The DNS resolution fails.
• Database blocked. The policy denies egress to the database Service. The application fails.
• Egress to external Internet blocked. The policy denies egress to external IPs. The application fails.
• kube-apiserver blocked. The policy denies egress to the API server. The kubelet integration fails.

flowchart TD
    A[DNS resolution failing] --> B{DNS allowed in policy?}
    B -->|No| C[Add DNS allow rule]
    B -->|Yes| D{kube-system reachable?}
    D -->|No| E[Fix kube-system namespace]
    D---|Yes| F[DNS Pods are failing]
    F --> G[Fix the DNS Pods]

The remediation

The remediation depends on the cause:

# Option 1: Add a DNS allowance
kubectl apply -f allow-dns.yaml

# Option 2: Add a database allowance
kubectl apply -f allow-database.yaml

# Option 3: Add an external Internet allowance
kubectl apply -f allow-external.yaml

# Option 4: Allow specific ports to the API server
kubectl apply -f allow-api-server.yaml

The remediation is the egress allowance.

The DNS allowance with port

The DNS allowance must include both UDP and TCP port 53:

egress:
- to:
  - namespaceSelector:
      matchLabels:
        kubernetes.io/metadata.name: kube-system
    podSelector:
      matchLabels:
        k8s-app: kube-dns
  ports:
  - protocol: UDP
    port: 53
  - protocol: TCP
    port: 53

The DNS allowance is the cluster’s DNS resolution.

Production discipline

Egress rules and DNS are the cluster’s network hypothesis. The discipline is to walk the canonical flow extended with the egress rules, identify the failure mode, apply the remediation. The network is the cluster’s connectivity; the remediation is the egress allowance.

• Allow DNS to kube-system. The DNS allowance is the cluster’s DNS resolution.
• Allow the database. The database allowance is the application’s data path.
• Allow the API server. The API server allowance is the cluster’s control plane.
• Allow external Internet if needed. The external Internet allowance is the application’s external path.

Quiz