eBPF mode — Cilium kube-proxy replacement and the kernel-bypass dataplane

Cilium replaces kube-proxy with eBPF. The eBPF dataplane is faster than iptables and IPVS, with no kernel rule overhead. The trade-off is the Cilium CNI dependency and the kernel version requirement. This lesson walks the eBPF mode, the requirements, and the operational discipline.

What eBPF replaces

The eBPF mode replaces the kube-proxy entirely. The cilium-agent on every node programs eBPF maps for the Service dataplane. The iptables and IPVS rules are not used.

flowchart LR
    A[Pod] -->|DNS query| B[CoreDNS]
    A -->|TCP to ClusterIP| C[eBPF datapath]
    C -->|DNAT to Pod IP| D[Pod]
    C -.programmed by.-> E[cilium-agent]
    E -.watches.-> F[Kubernetes API]

The cilium-agent watches the API and programs the eBPF maps. The eBPF maps are evaluated by the kernel on every packet; the evaluation is O(1).

The eBPF advantages

The eBPF mode has several advantages over iptables and IPVS:

• Performance: the eBPF dataplane is faster than iptables and IPVS. The packet is processed in the kernel without traversing the netfilter chain.
• Scalability: the eBPF maps scale to millions of endpoints without performance degradation.
• Observability: the eBPF dataplane exposes Hubble, Cilium’s observability platform. Hubble shows the flow logs in real time.
• Security: the eBPF dataplane supports CiliumNetworkPolicy, which can match on FQDN, HTTP method, gRPC service, and other Layer 7 attributes.

Adopting the eBPF mode means adopting Cilium as the CNI, and swapping the CNI on a running cluster means recycling every Pod so each one takes an address from the new IPAM.

The eBPF requirements

The eBPF mode has the following requirements:

• Cilium CNI 1.10+: the Cilium CNI must be installed.
• Linux kernel 5.4+: the kernel must support the basic eBPF features.
• Linux kernel 5.10+: the kernel must support the advanced eBPF features (e.g., TCP load balancing, FQDN policies).
• BPF mount: the kernel must have the BPF filesystem mounted at /sys/fs/bpf.

The cluster operator must verify the requirements before enabling the eBPF mode.

The cilium-agent’s configuration

The cilium-agent is configured to enable the kube-proxy replacement:

apiVersion: cilium.io/v2
kind: CiliumClusterwideNetworkPolicy
metadata:
  name: cilium-default
spec:
  nodeSelector: {}
  ingress: []
  egress: []

The cilium-agent’s configuration includes:

kubeProxyReplacement: strict
loadBalancer:
  algorithm: random

The strict value disables the kube-proxy entirely. The random algorithm is the default load balancing algorithm.

The Hubble observability

The eBPF mode includes Hubble, Cilium’s observability platform. Hubble provides real-time flow logs:

hubble observe

Jan  1 12:00:00.000  default/billing-7d4 -> prod-app/billing-8e5
  TCP FORWARDED (HTTP/200)

The Hubble flow logs are the cluster’s network telemetry. The cluster operator can use Hubble to diagnose network issues.

The failure modes

The eBPF mode’s failure modes:

• Kernel version too old: the kernel does not support the eBPF features. The fix is to upgrade the kernel.
• Cilium agent down: the cilium-agent is not running. The fix is to restart the agent.
• eBPF map corruption: the eBPF maps are corrupted. The fix is to restart the agent.
• Cilium CNI misconfigured: the Cilium CNI’s configuration is wrong. The fix is to update the configuration.
• Hubble down: the Hubble flow logs are not available. The fix is to verify the Hubble deployment.

The operational discipline

The eBPF mode’s operational discipline:

• Document the kube-proxy mode. The cluster operator must understand which mode is used.
• Verify the kernel version. The kernel must be 5.4+ for basic eBPF and 5.10+ for advanced.
• Verify the Cilium CNI version. The Cilium CNI must be 1.10+.
• Monitor the eBPF metrics. The eBPF metrics are the leading indicator.
• Test the mode in staging. The eBPF mode is critical infrastructure.
• Plan the mode’s evolution. The eBPF mode is the future of the Kubernetes Service dataplane.
• Document the mode’s design. The eBPF mode is the cluster’s Service dataplane; the documentation is the reference.

Quiz

Production discipline

• eBPF mode requires Cilium CNI and a recent kernel. The cluster operator must verify the requirements.
• Document the kube-proxy mode. The cluster operator must understand which mode is used.
• Verify the kernel version. The kernel must be 5.4+ for basic eBPF and 5.10+ for advanced.
• Verify the Cilium CNI version. The Cilium CNI must be 1.10+.
• Monitor the eBPF metrics. The eBPF metrics are the leading indicator.
• Test the mode in staging. The eBPF mode is critical infrastructure.
• Plan the mode’s evolution. The eBPF mode is the future of the Kubernetes Service dataplane.
• Document the mode’s design. The eBPF mode is the cluster’s Service dataplane; the documentation is the reference.
• Train the operations team on the eBPF diagnostics. The eBPF mode requires eBPF knowledge.