The Kubernetes networking model — four rules and a contract

The Kubernetes networking model is a contract between the cluster and the underlying network. The model specifies four rules that every cluster must implement. The model is implemented by the CNI plugin, which configures the cluster’s network to satisfy the rules. This lesson walks the four rules, the CNI plugin’s responsibility, and the operational patterns.

The four rules

The Kubernetes networking model’s four rules:

1. Every Pod gets its own IP address. The Pod’s IP is allocated by the cluster’s IPAM (IP Address Management).
2. Pods on any node can communicate with all other Pods on any node without NAT. The cluster’s network routes Pod-to-Pod traffic without address translation.
3. Agents on a node (the kubelet, the runtime, the CNI agent) can communicate with all Pods on all nodes without NAT. The cluster’s network routes agent-to-Pod traffic without address translation.
4. Pods in the host network can communicate with all Pods on all nodes without NAT. The cluster’s network routes host-network Pods to other Pods without address translation.

The four rules are the cluster’s networking contract. The CNI plugin is responsible for implementing the rules.

The first rule: every Pod gets an IP

The first rule is that every Pod gets its own IP address. The IP is allocated by the cluster’s IPAM.

flowchart LR
    A[Pod 1] -->|IP: 10.244.1.5| B[Pod network]
    C[Pod 2] -->|IP: 10.244.1.6| B
    D[Pod 3] -->|IP: 10.244.2.5| B
    E[Pod 4] -->|IP: 10.244.2.6| B

The Pod’s IP is unique within the cluster. No two Pods share an IP.

The IPAM is the cluster’s mechanism for allocating IPs. The IPAM is typically:

• Per-node CIDR: each node has a CIDR; the CNI allocates IPs from the node’s CIDR.
• Cluster-wide pool: the cluster has a single CIDR; the CNI allocates IPs from the cluster’s CIDR.
• IPAM plugin: the CNI uses an IPAM plugin (host-local, Calico IPAM, etc.).

The IPAM is the cluster’s mechanism for ensuring that every Pod has a unique IP.

The second rule: Pod-to-Pod communication

The second rule is that Pods on any node can communicate with all other Pods without NAT. The cluster’s network routes Pod-to-Pod traffic without address translation.

flowchart LR
    A[Pod 1<br/>10.244.1.5] -->|no NAT| B[Pod 2<br/>10.244.2.6]

The cluster’s network is a routed network. The Pod’s IP is routable from any node. The CNI configures the routes.

The routing is implemented by the CNI plugin:

• Bridge + route: the CNI creates a bridge on each node; the bridge is connected to the host network. The CNI adds routes for the Pod CIDRs.
• Overlay: the CNI creates an overlay network; the Pod traffic is encapsulated in the overlay.
• BGP: the CNI uses BGP to advertise the Pod CIDRs to the network.

The routing is the cluster’s networking enforcement of the second rule.

The third rule: agent-to-Pod communication

The third rule is that agents on a node can communicate with all Pods without NAT. The cluster’s network routes agent-to-Pod traffic without address translation.

flowchart LR
    A[kubelet on node-1] -->|no NAT| B[Pod on node-2]
    C[API server on node-3] -->|no NAT| B

The agents (the kubelet, the runtime, the CNI agent) on any node can communicate with all Pods on all nodes. The CNI configures the routes for the agent traffic.

The agent traffic is typically routed through the cluster’s overlay or the BGP. The CNI plugin is responsible for the routing.

The fourth rule: host-network Pods

The fourth rule is that Pods in the host network can communicate with all Pods without NAT. The cluster’s network routes host-network Pods to other Pods without address translation.

flowchart LR
    A[Host-network Pod<br/>node-1's IP] -->|no NAT| B[Pod on node-2<br/>10.244.2.6]

A Pod with hostNetwork: true shares the node’s network namespace. The Pod’s IP is the node’s IP. The cluster’s network must route traffic from the host’s IP to the Pod’s IP on the other node.

The CNI does not allocate an address for such a Pod; the node’s existing routes carry the traffic in both directions.

The CNI plugin’s responsibility

The CNI plugin is the cluster’s implementation of the networking model. The CNI plugin’s responsibilities:

• Pod sandbox creation: the CNI plugin creates the Pod’s network namespace.
• Interface attachment: the CNI plugin attaches the Pod’s interface (veth pair) to the host network.
• IP allocation: the CNI plugin allocates the Pod’s IP from the IPAM.
• Route configuration: the CNI plugin configures the routes for the Pod’s traffic.
• Network policy: the CNI plugin enforces the NetworkPolicy (if supported).

The CNI plugin is the cluster’s networking enforcement of the four rules.

The standard CNI plugins

The standard CNI plugins:

The production rule is to choose a CNI that implements the four rules correctly and supports the operational patterns (network policy, observability, troubleshooting).

The CNI plugin’s interfaces

The CNI plugin’s interfaces:

• CNI ADD: the kubelet calls the CNI plugin’s ADD command when the Pod’s sandbox is created. The CNI plugin attaches the Pod’s interface.
• CNI DEL: the kubelet calls the CNI plugin’s DEL command when the Pod’s sandbox is deleted. The CNI plugin detaches the Pod’s interface.
• CNI CHECK: the kubelet calls the CNI plugin’s CHECK command to verify the Pod’s network.

The CNI plugin is the cluster’s networking implementation. The kubelet is the orchestrator.

The CNI plugin’s configuration

The CNI plugin’s configuration is in /etc/cni/net.d/:

ls /etc/cni/net.d/

10-calico.conflist

The configuration file is a JSON or YAML file that specifies the CNI plugin’s parameters. The kubelet reads the file and calls the CNI plugin.

The standard configuration:

{
  "name": "k8s-pod-network",
  "cniVersion": "1.0.0",
  "plugins": [
    {
      "type": "calico",
      "log_level": "info",
      "datastore_type": "kubernetes",
      "policy": {
        "type": "k8s"
      },
      "kubernetes": {
        "kubeconfig": "/etc/cni/net.d/calico-kubeconfig"
      }
    },
    {
      "type": "portmap",
      "capabilities": {
        "portMappings": true
      }
    }
  ]
}

The configuration is the cluster’s networking configuration. The CNI plugin reads the configuration on startup.

The CNI plugin’s troubleshooting

The CNI plugin’s troubleshooting:

journalctl -u kubelet | grep -i cni

The kubelet’s logs show the CNI plugin’s errors. The fix is to investigate the CNI plugin’s logs.

The CNI plugin’s troubleshooting:

# Substitute your own value before running:
POD=web-5f9c7d8b6c-2xk9p

# Check the Pod's network namespace
kubectl exec "$POD" -- ip addr show

The command shows the Pod’s network interfaces. The fix is to investigate the Pod’s network.

The CNI plugin’s troubleshooting:

# Check the node's routes
ip route

The command shows the node’s routes. The fix is to investigate the node’s routes.

Quiz

Production discipline

• The Kubernetes networking model is a contract. The four rules are the cluster’s guarantees.
• The CNI plugin implements the four rules. The CNI plugin is the cluster’s networking implementation.
• Choose a CNI that supports the operational patterns. The CNI should support network policy, observability, and troubleshooting.
• Audit the CNI plugin’s configuration at every release. The CNI plugin’s configuration should be version-controlled; the audit catches the failures.
• Test the CNI plugin in non-production. A staging cluster that mirrors production is the right place to test the CNI plugin.
• Monitor the CNI plugin’s metrics. The CNI plugin’s metrics expose the cluster’s networking health; the operator should alert on the threshold.
• Document the CNI plugin’s design. The CNI plugin is the cluster’s networking implementation; the documentation is the cluster’s networking reference.