CNI chaining — delegating to multiple plugins in order

CNI chaining lets the kubelet invoke multiple plugins in a single ADD call. The conflist declares a list of plugins; libcni invokes them in order, passing the result of each plugin to the next as the prevResult field. DEL reverses the order. This lesson walks the chain contract, the built-in plugins, and the operational discipline of designing a chain.

The conflist format

The conflist is a JSON file with a plugins list:

{
  "name": "k8s-pod-network",
  "cniVersion": "1.0.0",
  "plugins": [
    {
      "type": "calico",
      "log_level": "info",
      "datastore_type": "kubernetes",
      "policy": { "type": "k8s" },
      "kubernetes": {
        "kubeconfig": "/etc/cni/net.d/calico-kubeconfig"
      }
    },
    {
      "type": "portmap",
      "capabilities": {
        "portMappings": true
      }
    },
    {
      "type": "bandwidth",
      "capabilities": {
        "bandwidth": true
      }
    }
  ]
}

The first plugin (Calico) is the main plugin. It configures the network namespace and produces an IP. The second plugin (portmap) sees the Calico result and adds host-port mappings. The third plugin (bandwidth) sees the combined result and applies traffic shaping.

sequenceDiagram
    autonumber
    participant L as libcni
    participant C as Calico
    participant P as portmap
    participant B as bandwidth
    L->>C: ADD (no prevResult)
    C-->>L: result IP/iface
    L->>P: ADD (prevResult)
    P-->>L: result + portmap
    L->>B: ADD (prevResult)
    B-->>L: result + bandwidth
    L-->>L: merge results

The merge combines the IPs, interfaces, and routes of every plugin. The kubelet sees a single result.

The reverse on DEL

DEL reverses the chain. The kubelet calls each plugin in reverse order; the prevResult is the merged result of the ADD call. The reverse order ensures that the cleanup is symmetric to the setup.

sequenceDiagram
    autonumber
    participant L as libcni
    participant C as Calico
    participant P as portmap
    participant B as bandwidth
    L->>B: DEL (prevResult)
    L->>P: DEL (prevResult)
    L->>C: DEL (prevResult)

If a plugin in the middle of the chain fails on DEL, the plugins that precede it are still called. The IPAM may leak; the next plugin’s cleanup still runs.

The built-in plugins

The CNI plugins tarball ships several built-in plugins that are designed to be chained with a main plugin:

The main plugin (Calico, Cilium, Flannel) typically requires loopback to be in the chain. The configuration is usually:

{
  "plugins": [
    { "type": "loopback" },
    { "type": "calico" },
    { "type": "portmap" },
    { "type": "bandwidth" }
  ]
}

The chain and NetworkPolicy

When the main plugin supports NetworkPolicy (Calico, Cilium), the chain does not include a separate policy plugin. The main plugin handles the policy on the host’s iptables or eBPF. The chain is for connectivity only; the policy is enforced by the main plugin.

A cluster that uses Flannel (no policy) and wants NetworkPolicy must add a policy plugin to the chain. The Calico policy plugin can be chained with Flannel:

{
  "plugins": [
    { "type": "flannel" },
    { "type": "calico", "policy": { "type": "k8s" } }
  ]
}

This is the standard “Flannel + Calico policy” pattern for clusters that want a simple dataplane with policy enforcement.

The chain and Multus

Multus is a CNI plugin that invokes other CNI plugins as delegates. Multus itself is the main plugin in the kubelet’s conflist; it reads the Pod’s network attachment definitions and invokes the appropriate sub-plugins. A Pod with multiple network interfaces uses Multus to chain the sub-plugins.

{
  "name": "multus-cni",
  "type": "multus",
  "capabilities": {
    "namespace": true
  },
  "delegates": [
    {
      "name": "k8s-pod-network",
      "cniVersion": "1.0.0",
      "plugins": [
        { "type": "loopback" },
        { "type": "calico" }
      ]
    }
  ]
}

Multus is the canonical example of a delegating plugin. The pattern is general: any CNI plugin can delegate to others.

The failure modes

The chain’s failure modes:

• Missing plugin binary: the conflist references a plugin that is not on the node. The kubelet retries ADD until the binary is installed.
• Wrong chain order: the loopback plugin is missing or placed after the main plugin. The Pod’s loopback is broken.
• Plugin exit non-zero: the entire ADD fails. Pods stay pending.
• Stale cache: the result cache from a previous chain is read by DEL. The plugin may fail to clean up.

The operational discipline

The chain’s operational discipline:

• Document the chain. The chain is the cluster’s networking implementation; the documentation is the reference.
• Version the conflist. The conflist is part of the cluster’s GitOps.
• Audit the chain order across nodes. A drift produces inconsistent Pod networking.
• Test the chain in staging. A misconfigured chain blocks every Pod on the node.
• Monitor the kubelet’s CNI logs. The logs show each plugin’s invocation and result.

Quiz

Production discipline

• The conflist is a chain. The order matters.
• Chain reversal on DEL. The cleanup is symmetric to the setup.
• Built-in plugins are auxiliary. They extend the main plugin’s capability.
• The chain is per-node. A drift across nodes produces inconsistent Pod networking.
• Version the conflist in GitOps. The chain is part of the cluster’s configuration.
• Audit the chain at every release. A drift catches the failures.
• Test the chain in staging. A misconfigured chain blocks every Pod on the node.
• Document the chain. The chain is the cluster’s networking implementation; the documentation is the reference.