Dependencies and component identification — the network of cause

Dependencies are the network of cause. The component is the one named piece of software — a controller, a CNI plugin, a kubelet, an application container — that the evidence points at. Steps 6 and 7 narrow the blast radius to a single suspect before step 8 asks why it failed. The discipline is to map the dependencies before identifying the component, because the component is often a dependent of the misbehaving object, not the object itself.

Step 6: Dependencies

The dependencies are the other objects the misbehaving object relies on. The dependencies are the cluster’s relationships: the Service that routes to the Pod, the EndpointSlice that lists the Pods, the ConfigMap that the Pod mounts, the Secret that the Pod reads, the PersistentVolume that the Pod claims.

flowchart TD
    A[Pod] --> B[Service]
    A --> C[EndpointSlice]
    A --> D[ConfigMap]
    A --> E[Secret]
    A --> F[PVC]
    A --> G[ServiceAccount]
    A --> H[NetworkPolicy]
    A --> I["Pod (peer)"]
    B --> C

The dependencies are the network of cause. A misbehaving Pod may be failing because:

• The Service has no endpoints (the selector is wrong).
• The EndpointSlice is empty (the pods are not Ready).
• The ConfigMap is missing (the mount is failing).
• The Secret is missing (the envFrom is failing).
• The PVC is Pending (the CSI is not provisioning).
• The ServiceAccount is forbidden (the RBAC is wrong).
• The NetworkPolicy is blocking egress (the policy is wrong).
• The peer Pod is unreachable (the application is wrong).

The dependencies are the input to step 7.

kubectl get pod billing-7d8f-abcde -n prod -o json | \
  jq '.spec.volumes[]?, .spec.containers[].env[]?, .spec.containers[].envFrom[]?'

The output is the list of dependencies. The operator walks the list and checks each one.

Step 7: Identify the component

The component is the piece of software that is failing. The component is not the cluster; the component is the Kubernetes control plane, the kubelet, the CNI, the CSI, the workload, or the user’s application. The identification is the moment when the operator decides which component is the source of the symptom.

flowchart TD
    A[Symptom] --> B{Where?}
    B -->|Pod| C[Workload, container]
    B -->|Node| D[Kubelet, runtime, kernel]
    B -->|Service| E[Service, EndpointSlice, kube-proxy]
    B -->|Control plane| F[API server, scheduler, controller]
    B -->|Storage| G[CSI, PV, PVC]
    B -->|Network| H[CNI, DNS, NetworkPolicy]

The identification is the input to step 8. The identification is the moment when the operator stops gathering evidence and proposes a candidate.

The identification is not a guess. The identification is the conclusion of the evidence-gathering arc. The operator who identifies the component without the arc is the operator who applies the wrong fix.

How to identify the component

The component is identified by the evidence. The most common identification patterns:

• Pod level. The Pod is in CrashLoopBackOff, the events show Back-off restarting failed container, the previous logs show the application’s crash. The component is the container (image, startup, probe).
• Node level. The node is NotReady, the events show NodeNotReady, the kubelet logs show the failure. The component is the kubelet (heartbeat, runtime, pressure).
• Service level. The Service returns 503, the EndpointSlice is empty, the synthetic traffic fails. The component is the Service (selector, port) or the kube-proxy (iptables, IPVS).
• Control plane level. The API server returns 503, the scheduler logs show the failure. The component is the API server (etcd, RBAC) or the scheduler (filter, score).
• Storage level. The PVC is Pending, the CSI logs show the failure. The component is the CSI driver (provisioner, attacher).
• Network level. The synthetic traffic fails, the NetworkPolicy is denying. The component is the CNI (datapath) or the NetworkPolicy (selector).

Failure modes

The most common failure modes:

• Fixing the wrong component. The operator identifies the workload as the failing component but the actual component is the dependency (e.g., the Service’s selector is wrong).
• Stopping at the first candidate. The operator sees the workload’s logs and assumes the workload is failing. The actual cause is the kubelet’s pressure.
• Ignoring the cluster’s view. The operator focuses on the application’s logs and misses the events that the cluster has already produced.

The discipline is to walk the dependencies, identify the component with the evidence, and document the choice in the PIR.

Production discipline

The dependencies are the network of cause. The component is the piece of software that is failing. The two are the input to step 8: the hypothesis. The discipline is to map the dependencies first, then identify the component, then propose the hypothesis.

• Walk the list. Service, EndpointSlice, ConfigMap, Secret, PVC, ServiceAccount, NetworkPolicy.
• Never fix the misbehaving object without checking the dependencies. The fix is often the dependency.

Quiz