CNI installation and upgrade — the operational discipline

A CNI is installed in two parts: the agent (a DaemonSet that runs on every node) and the conflist (a file on every node’s /etc/cni/net.d/). The kubelet reads the conflist on every ADD; the agent configures the host networking. This lesson walks the installation, the upgrade, and the operational discipline of treating the CNI as critical infrastructure.

The two parts of a CNI

Every CNI has two parts:

The agent is the control plane: it watches the Kubernetes API, manages the IPAM store, and configures the BGP or overlay. The conflist is the data plane contract: the kubelet invokes the plugin with the conflist’s contents.

flowchart LR
    A[CNI agent DaemonSet] -->|configures| B[Host networking]
    A -->|watches| C[Kubernetes API]
    D[conflist on node] -->|read by| E[kubelet]
    E -->|invokes| F[CNI plugin ADD/DEL]
    F -->|mutates| G[Pod netns]

The conflist is the only file the kubelet requires. The agent is optional for some CNIs (Flannel runs the agent as a DaemonSet, but the conflist is the only thing the kubelet reads).

The kubeadm discovery path

The kubelet, on startup, reads /etc/cni/net.d/ and picks the first conflist it finds. The kubelet does not validate the conflist; it trusts the operator to install the right one.

ls /etc/cni/net.d/

10-calico.conflist
calico-kubeconfig

The convention is 10-<name>.conflist where the leading number is the priority. The kubelet picks the first file matching *.conflist or *.conf in lexicographic order.

Installing Calico on kubeadm

The standard install pattern is:

# Download the Calico manifest
curl -L https://raw.githubusercontent.com/projectcalico/calico/v3.28.0/manifests/calico.yaml -o calico.yaml

# Apply the manifest
kubectl apply -f calico.yaml

The manifest installs:

• The calico-node DaemonSet (one Pod per node).
• The calico-kube-controllers Deployment (cluster-wide reconciliation).
• The calico-cni-plugin binary on every node (via the DaemonSet’s hostPath mount).
• The 10-calico.conflist on every node (via the DaemonSet’s hostPath mount).

The upgrade is a rolling replace of the DaemonSet. The conflist is re-applied on every node as the new DaemonSet’s Pod touches the hostPath.

The DaemonSet pattern

The CNI agent is a DaemonSet because the agent must run on every node. The DaemonSet’s Pod manifest typically:

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: calico-node
  namespace: kube-system
spec:
  selector:
    matchLabels:
      k8s-app: calico-node
  template:
    metadata:
      labels:
        k8s-app: calico-node
    spec:
      hostNetwork: true
      containers:
        - name: calico-node
          image: quay.io/calico/node:v3.28.0
          env:
            - name: DATASTORE_TYPE
              value: kubernetes
          volumeMounts:
            - mountPath: /host/etc/cni/net.d
              name: cni-net-dir
            - mountPath: /host/opt/cni/bin
              name: cni-bin-dir
      volumes:
        - name: cni-net-dir
          hostPath:
            path: /etc/cni/net.d
        - name: cni-bin-dir
          hostPath:
            path: /opt/cni/bin

The hostPath mounts are the mechanism by which the DaemonSet installs the conflist and binaries on every node. The agent’s init container (or the apply path in the bootstrap) writes the files.

The upgrade cycle

The CNI upgrade cycle:

1. Read the release notes. The release notes document breaking changes, deprecated flags, and known issues.
2. Test the upgrade in staging. The CNI is critical infrastructure; the upgrade must be promoted through staging.
3. Apply the new manifest. The manifest updates the DaemonSet; Kubernetes performs a rolling replace.
4. Verify the new Pods are Ready. The new agent must be Ready before the old Pod is removed.
5. Verify the conflist is updated. The new conflist must be present on every node.
6. Verify the cluster is healthy. The CNI’s metrics (IPAM, BGP, etc.) must show the new version.

The failure modes

The install and upgrade failure modes:

• Conflist missing: the kubelet cannot schedule Pods. The fix is to apply the manifest.
• Binary missing: the kubelet’s CNI calls fail. The fix is to verify the hostPath mount.
• DaemonSet stuck: the new agent cannot roll. The fix is to check the kubelet’s logs and the agent’s logs.
• Conflicting conflists: two conflists with overlapping priorities. The fix is to delete the stale one.
• Version skew: the CNI version does not match the Kubernetes version. The fix is to read the compatibility matrix.

The operational discipline

The CNI install and upgrade’s operational discipline:

• Pin the CNI version in GitOps. The manifest is part of the cluster’s configuration.
• Test the upgrade in staging. The CNI is critical infrastructure.
• Audit the conflist at every release. A drift blocks every Pod on the node.
• Monitor the CNI’s metrics. The CNI’s metrics are the leading indicator of failure.
• Document the install path. The CNI is the cluster’s networking implementation; the documentation is the reference.
• Plan the CNI’s exit. The CNI is the cluster’s most locked-in dependency; the exit must be planned.

Quiz

Production discipline

• The CNI is the cluster’s most critical infrastructure. The install and upgrade must be performed with the same discipline as the kubelet.
• The CNI has two parts. The agent DaemonSet and the conflist; both must be present and correct.
• Pin the CNI version in GitOps. The manifest is part of the cluster’s configuration.
• Test the upgrade in staging. The CNI is critical infrastructure; promote through staging.
• Audit the conflist at every release. A drift blocks every Pod on the node.
• Monitor the CNI’s metrics. The CNI’s metrics are the leading indicator of failure.
• Document the install path. The CNI is the cluster’s networking implementation; the documentation is the reference.
• Plan the CNI’s exit. The CNI is the cluster’s most locked-in dependency; the exit must be planned.