MTU and the CNI — how Calico, Cilium, Flannel set the Pod MTU

The CNI’s MTU configuration is one of the most common misconfigurations in a Kubernetes cluster. The defaults are correct for the standard 1500-byte underlay with VXLAN; they are wrong for any other configuration. This lesson walks the configuration for each major CNI.

Calico

Calico exposes the Pod MTU through the Installation custom resource. The default is 1450 (assuming VXLAN on 1500). For IPIP, the default is 1480. For routed (BGP), the default is 1500.

# Inspect the Calico Installation
kubectl get installation default -o yaml

apiVersion: operator.tigera.io/v1
kind: Installation
metadata:
  name: default
spec:
  calicoNetwork:
    ipPools:
    - blockSize: 26
      cidr: 10.244.0.0/16
      encapsulation: VXLAN
      natOutgoing: Enabled
    mtu: 1450   # <-- Pod eth0 MTU

To change the MTU:

kubectl patch installation default --type=merge \
  -p '{"spec":{"calicoNetwork":{"mtu":1480}}}'

The change requires a rolling restart of the Calico Pods. The new MTU applies to Pods created after the restart; existing Pods keep their old MTU until they are recreated.

# Substitute your own values before running:
POD=web-5f9c7d8b6c-2xk9p   # a Pod created after the Calico restart
VETH=cali7f3a9b2c1d4       # its host-side veth, from `ip link` on the node

# Verify the new MTU is applied
kubectl exec "$POD" -- ip link show eth0
# 4: eth0@if12345: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1480 ...

# Check the host-side veth MTU
ip link show "$VETH"
# mtu 1480

Cilium

Cilium exposes the MTU through the cilium-config ConfigMap in kube-system. The relevant keys:

• cilium_mtu: the Pod eth0 MTU (default 1450 for VXLAN).
• geneve_mtu: the MTU for Geneve overlays.
• ipv4_mtu, ipv6_mtu: per-family MTU overrides.

kubectl -n kube-system get configmap cilium-config -o yaml

data:
  cilium_mtu: "1450"
  enable_ipv4: "true"
  enable_ipv6: "true"
  tunnel: vxlan
  # ...

To change the MTU, edit the ConfigMap and restart every Cilium agent:

kubectl -n kube-system edit configmap cilium-config
# Set cilium_mtu: "1480"

# Restart the Cilium agents
kubectl -n kube-system rollout restart daemonset cilium

The Cilium DaemonSet’s pods roll one node at a time; during the roll, the cluster has mixed MTUs (some Pods on old nodes have 1450, some on new nodes have 1480). Connectivity during the roll works because the lower MTU is the constraint (a Pod sending 1450-byte packets to a Pod with 1480-byte MTU works; the reverse may fragment).

Flannel

Flannel’s MTU is configured in the kube-flannel-cfg ConfigMap in kube-system:

kubectl -n kube-system get configmap kube-flannel-cfg -o yaml

{
  "Network": "10.244.0.0/16",
  "Backend": {
    "Type": "vxlan",
    "MTU": 1450
  }
}

To change the MTU:

kubectl -n kube-system edit configmap kube-flannel-cfg
# Set Backend.MTU to 1480

# Restart the Flannel DaemonSet
kubectl -n kube-system rollout restart daemonset kube-flannel-ds

Flannel’s MTU setting is straightforward; the change is a ConfigMap edit and a DaemonSet rollout.

Weave Net

Weave Net is configured via the Weave Pod’s command-line arguments:

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: weave-net
  namespace: kube-system
spec:
  template:
    spec:
      containers:
      - name: weave
        env:
        - name: MTU
          value: "1410"  # <-- Weave MTU

Weave uses an estimate of the underlay MTU and subtracts the Weave overhead. The default 1410 assumes a 1500-byte underlay.

To change the Weave MTU, edit the DaemonSet and update the MTU environment variable, then roll the DaemonSet.

Multus and secondary interfaces

Multus allows Pods to have multiple network interfaces, each attached to a different CNI. The MTU must be set per network attachment definition:

apiVersion: k8s.cni.cncf.io/v1
kind: NetworkAttachmentDefinition
metadata:
  name: macvlan-conf
spec:
  config: '{
    "type": "macvlan",
    "master": "eth0",
    "ipam": {
      "type": "whereabouts",
      "range": "192.168.1.0/24"
    },
    "mtu": 1500
  }'

The MTU for the secondary interface is set in the NetworkAttachmentDefinition. Multus does not enforce consistency between the primary and secondary MTUs; the operator must.

Verifying cluster-wide MTU

The standard validation:

# Sample MTU across nodes
for node in $(kubectl get nodes -o name); do
  echo "=== $node ==="
  kubectl debug node/${node#node/} -it --image=nicolaka/netshoot \
    --target=node -- ip link show cni0 2>/dev/null || \
    kubectl debug node/${node#node/} -it --image=nicolaka/netshoot \
    -- ip link show cni0
done

# Sample MTU across Pods
for pod in $(kubectl get pods -A -o name | head -20); do
  echo "=== $pod ==="
  kubectl exec ${pod#pod/} -c ${pod##*/} -- ip link show eth0 2>/dev/null
done

A cluster where the Pod MTU is inconsistent across nodes has a misconfigured CNI. The fix is to identify the node group with the wrong MTU, check the CNI’s per-node configuration, and reconcile.

Quiz

Production discipline

• Document the CNI’s MTU setting in the cluster bootstrap. The default is correct for a standard underlay; the cluster operator must override it for any non-standard underlay.
• Validate the MTU cluster-wide before declaring the cluster ready. A cluster with mixed MTUs has connectivity problems that manifest as intermittent failures.
• Test the MTU against the actual underlay. A site-to-site VPN, an IPSec tunnel, or a constrained cloud underlay changes the calculation.
• MTU changes require a rolling restart of the CNI and a recreate of the workloads. The change is not instantaneous.