NetworkPolicy egress rules — controlling outbound traffic

NetworkPolicy egress rules control outbound traffic. The default-deny egress blocks all traffic, including DNS and kube-apiserver. The cluster operator must add explicit allow rules for DNS, kube-apiserver, and external services. This lesson walks the egress rules, the DNS and kube-apiserver allow rules, and the operational discipline.

The egress rules

The egress rules allow traffic to a set of destinations:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: prod-app-allow-egress
  namespace: prod-app
spec:
  podSelector:
    matchLabels:
      app: billing
  policyTypes:
    - Egress
  egress:
    - to:
        - podSelector:
            matchLabels:
              app: database
      ports:
        - protocol: TCP
          port: 5432
    - to:
        - namespaceSelector:
            matchLabels:
              kubernetes.io/metadata.name: kube-system
          podSelector:
            matchLabels:
              k8s-app: kube-dns
      ports:
        - protocol: UDP
          port: 53

The egress rules allow traffic to the database and to the kube-dns Service.

The default-deny egress

The default-deny egress blocks all traffic:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: prod-app-default-deny-egress
  namespace: prod-app
spec:
  podSelector: {}
  policyTypes:
    - Egress

The podSelector: {} matches every Pod in the namespace. The policyTypes: Egress declares egress deny. The result is that every Pod in the namespace has no egress by default.

The DNS egress rule

The DNS egress rule allows DNS resolution:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: prod-app-allow-dns
  namespace: prod-app
spec:
  podSelector: {}
  policyTypes:
    - Egress
  egress:
    - to:
        - namespaceSelector:
            matchLabels:
              kubernetes.io/metadata.name: kube-system
          podSelector:
            matchLabels:
              k8s-app: kube-dns
      ports:
        - protocol: UDP
          port: 53
        - protocol: TCP
          port: 53

The DNS egress rule allows TCP and UDP port 53 to the kube-dns Service. Without this rule, the Pods cannot resolve DNS names.

The kube-apiserver egress rule

The kube-apiserver egress rule allows API server access:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: prod-app-allow-api
  namespace: prod-app
spec:
  podSelector: {}
  policyTypes:
    - Egress
  egress:
    - to:
        - ipBlock:
            cidr: 10.96.0.1/32
      ports:
        - protocol: TCP
          port: 443

The kube-apiserver egress rule allows TCP 443 to the API server’s IP. The API server’s IP is the kubernetes.default.svc.cluster.local ClusterIP.

The external egress rule

The external egress rule allows external traffic:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: prod-app-allow-external
  namespace: prod-app
spec:
  podSelector:
    matchLabels:
      app: egress-proxy
  policyTypes:
    - Egress
  egress:
    - to:
        - ipBlock:
            cidr: 0.0.0.0/0
            except:
              - 10.0.0.0/8
              - 172.16.0.0/12
              - 192.168.0.0/16
      ports:
        - protocol: TCP
          port: 443

The external egress rule allows HTTPS to the internet but blocks RFC 1918. The use case is a controlled egress proxy.

The failure modes

The egress rules’ failure modes:

• Default-deny too strict: the default-deny blocks DNS or kube-apiserver. The fix is to add explicit allow rules.
• DNS egress missing: the Pods cannot resolve DNS. The fix is to add the DNS egress rule.
• API egress missing: the Pods cannot authenticate to the API server. The fix is to add the API egress rule.
• External egress too permissive: the egress rule allows traffic to unintended IPs. The fix is to tighten the ipBlock.

The operational discipline

The egress rules’ operational discipline:

• Document the egress rules. The rules are the cluster’s firewall.
• Add the DNS allow rule for the default-deny. The Pods must be able to resolve DNS.
• Add the API allow rule for the default-deny. The Pods must be able to authenticate to the API server.
• Audit the egress rules at every change. The rules are critical security configuration.
• Test the egress rules in staging. The rules must work for the workload.
• Monitor the egress rules’ metrics. The CNI exposes NetworkPolicy metrics.
• Use the default-deny pattern. The default-deny is the basis of zero-trust.

Quiz

Production discipline

• The egress rules are the cluster’s firewall. The cluster operator must treat them as critical security configuration.
• Document the egress rules. The rules are the cluster’s firewall.
• Add the DNS allow rule for the default-deny. The Pods must be able to resolve DNS.
• Add the API allow rule for the default-deny. The Pods must be able to authenticate to the API server.
• Audit the egress rules at every change. The rules are critical security configuration.
• Test the egress rules in staging. The rules must work for the workload.
• Monitor the egress rules’ metrics. The CNI exposes NetworkPolicy metrics.
• Use the default-deny pattern. The default-deny is the basis of zero-trust.
• Document the troubleshooting. The troubleshooting is the cluster’s operational reference.