CNI-aware capture — Calico, Cilium, and the tools that know the cluster

A raw tcpdump does not know what a Pod is, what a NetworkPolicy is, or which Kubernetes workload is the source of a packet. CNI-aware tools do. This lesson walks the CNI-specific capture tooling: Calico’s flow logs, Cilium’s Hubble, and the kubectl plugin ecosystem.

Why CNI-aware tools matter

A tcpdump capture gives you bytes. A CNI-aware capture gives you:

• The source and destination as Kubernetes identities (Pod name, namespace, labels), not just IP addresses.
• The NetworkPolicy that allowed or dropped the flow.
• The Service the traffic was destined for (post-DNAT).
• The DNS name the application tried to resolve.
• The TLS SNI the client sent.

This metadata is the difference between “I see a SYN to 10.244.5.7 from 10.244.1.12” and “I see a SYN from app-a.production to api.production that was allowed by the allow-internal-networkpolicy and routed to api-7c8f2d8e.” The second is a debugging fact; the first is a forensics task.

Calico flow logs and Felix logs

Calico emits structured flow logs via Felix (the per-node dataplane). The logs are configured per-StorageClass analogue — per-FelixConfiguration:

apiVersion: projectcalico.org/v3
kind: FelixConfiguration
metadata:
  name: default
spec:
  flowLogsEnabled: true
  flowLogsFileDirectory: /var/log/calico/flows
  flowLogsFlushInterval: 5s
  flowLogsCollectorDaisyChain: ...

Once enabled, Felix writes one structured event per flow (start, end, denied):

{
  "start_time": "2026-08-16T13:42:01.123Z",
  "end_time": "2026-08-16T13:42:11.456Z",
  "action": "allow",
  "src_ip": "10.244.1.5",
  "dst_ip": "10.244.2.6",
  "src_port": 42184,
  "dst_port": 8080,
  "src_name": "app-a.production",
  "dst_name": "api.production",
  "src_namespace": "production",
  "dst_namespace": "production",
  "src_labels": {"app":"app-a"},
  "dst_labels": {"app":"api"},
  "protocol": "tcp",
  "reporter": "src",
  "policies": ["allow-internal"]
}

This is a single event that answers: who, what, when, what policy, what namespace, what label. A tcpdump cannot produce this without external lookup.

calicoctl and the policy model

Calico has its own CLI, calicoctl, that exposes the policy model directly:

# Show all NetworkPolicies
calicoctl get networkpolicy -A

# Show the policy endpoints (which Pods the policy applies to)
calicoctl get policyendpoint -A

# Show the Felix log level (for debugging the dataplane)
calicoctl get felixconfiguration default -o yaml

# Show the BGP status (for routed clusters)
calicoctl get bgppeers
calicoctl get bgpstatus

For “the policy is blocking my traffic,” the standard diagnostic:

# Substitute your own values before running:
POD=app-a-7c8f2d8e
POLICY=allow-app-a-to-api
NS=production

# Check whether the policy applies
calicoctl get policyendpoint -o yaml | grep -B2 -A20 "$POD"

# Check whether the policy allows the traffic
calicoctl get networkpolicy "$POLICY" -n "$NS" -o yaml

A policy that does not have the right selector does not match the Pod; a selector that matches the Pod but has the wrong rules denies the traffic. Both failures are visible in calicoctl and in the flow logs.

Cilium Hubble

Cilium’s Hubble is the eBPF-based observability platform for Kubernetes. It provides L3/L4/L7 visibility with full Kubernetes identity context.

# Observe all flows in a namespace
hubble observe --namespace production

# Observe HTTP/gRPC flows (L7)
hubble observe --namespace production --protocol http

# Observe flows with a verdict (allowed, dropped, error)
hubble observe --namespace production --verdict DROPPED

# Observe flows to/from a specific Pod
hubble observe --pod app-a

# Follow flows in real time
hubble observe --follow

The output is structured:

Aug 16 13:42:01.123456  default production  app-a-7c8f2d8e  api-3a9b4c1d
  HTTP/1.1 GET http://api.production/users/42
  -> FORWARDED (HTTP 200, 12ms)

Aug 16 13:42:02.456789  default production  app-a-7c8f2d8e  api-3a9b4c1d
  HTTP/1.1 POST http://api.production/login
  -> DROPPED (Policy denied)

The DROP line tells the operator exactly which flow was denied, by which policy, with full identity context.

# Hubble UI (browser-based)
hubble ui --port 12000

# Hubble Relay (cluster-wide aggregation)
kubectl -n kube-system get pods -l k8s-app=hubble-relay

The Hubble Relay aggregates flow events from every node’s Hubble agent. For cluster-wide visibility, the Relay is required; the per-node Hubble only sees flows on its own node.

Weave Net scope

Weave Net includes a Scope product for visualization (not strictly packet capture, but cluster-aware network observability):

# Install Weave Scope
kubectl apply -f "https://cloud.weave.works/launch/k8s/weavescope.yaml"

# Access the UI
kubectl port-forward -n weave "$(kubectl get pod -n weave --selector=weave-scope-component=app -o jsonpath='{.items..metadata.name}')" 4040

Scope shows a topology view: Pods, Services, Nodes, and the connections between them. For “what is talking to what,” Scope is faster than flow logs. For “what is in the bytes,” Scope is not enough.

kubectl plugin ecosystem

The Kubernetes plugin ecosystem includes several CNI-aware capture tools:

# Substitute your own values before running:
POD=app-a-7c8f2d8e
NS=production

# ksniff: capture from a remote Pod via kubectl
kubectl sniff "$POD" -n "$NS" -o /tmp/cap.pcap

ksniff (by Eldad Rudich) is a kubectl plugin that copies tcpdump into the target Pod, runs it, streams the pcap to the operator’s local machine, and cleans up. It is the production default for “I want to tcpdump a Pod without installing anything on the cluster.”

# Substitute your own value before running:
POD=app-a-7c8f2d8e

# kubectl-trace: eBPF-based tracing
kubectl-trace --pod "$POD" --selector app=app-a \
              --probe 'kprobe:tcp_connect { printf("conn %p\\n", arg0); }'

kubectl-trace runs eBPF probes against a Pod’s kernel. For low-level kernel tracing (where packets are dropped, which socket buffer is full), it is more useful than tcpdump.

# netshoot: a debug image with all the tools
kubectl run netshoot --rm -it --image=nicolaka/netshoot -- bash

netshoot is the standard debug image. It includes tcpdump, tshark, curl, dig, mtr, iperf3, iftop, nmap, and dozens more. The kubectl run form creates an ephemeral debug Pod with all the tools.

Choosing between tools

The standard ladder: identity-aware tooling first (Hubble, flow logs), wire-level capture last. Operators who reach for tcpdump first spend hours assembling what Hubble would have shown in seconds.

Quiz

Production discipline

• Default to CNI-aware tools. Hubble, Calico flow logs, Weave Scope answer most cluster questions without wire capture.
• Enable flow logs cluster-wide. They are cheap relative to the diagnostic value, and the operator needs them at 3 AM.
• Treat wire-level capture as a last resort. Use it only when the identity-aware tooling says “the bytes are the problem.”
• Ship flow logs to your log aggregator. Loki, ELK, or the cluster’s standard sink. Per-node files are not queryable at cluster scale.
• Prefer ksniff over installing tcpdump on the cluster. It is the lowest-friction path to a Pod-level capture with no cluster changes.