Selector mismatch diagnosis — the policy that misses the Pod

A NetworkPolicy fails open. If its podSelector matches no Pods — because a label was renamed in the Deployment’s template, or the policy was applied to the wrong namespace — the traffic it was written to block flows exactly as it did before. The mismatch is usually one word, and this lesson covers proving what a policy actually selects against what the Pods actually carry.

The selector

A NetworkPolicy’s spec.podSelector selects the Pods that the policy applies to. The CNI’s policy enforcement watches the policy and the Pods’ labels and applies the policy to the matching Pods.

flowchart TD
    A[NetworkPolicy] --> B{podSelector matches Pods?}
    B -->|Yes| C[Policy applied]
    B -->|No| D[Policy not applied]

A selector mismatch is one where the policy’s selector does not match the Pods. The policy is not applied.

The diagnostic

The canonical diagnostic:

# Substitute your own values before running:
POLICY=billing-allow-ingress
NS=prod
SOURCE_POD=billing-7d8f-abcde
TARGET_IP=192.0.2.45
TARGET_PORT=8080

# 1. Check the NetworkPolicy's selector
kubectl get networkpolicy "$POLICY" -n "$NS" -o yaml

# 2. Check the Pod's labels
kubectl get pods -n "$NS" --show-labels

# 3. Check the policy's namespace
kubectl get networkpolicy -A

# 4. Test the connection
kubectl exec -it "$SOURCE_POD" -- curl -v "$TARGET_IP:$TARGET_PORT"

The diagnostic is the policy’s selector and the Pod’s labels.

Common causes of selector mismatch

The most common causes:

• Different label. The policy’s selector requires app=billing but the Pods have app=billing-api.
• Missing label. The Pods are missing a label that the selector requires.
• Namespace mismatch. The policy is in a different namespace than the Pods.
• Renamed label. The Pod’s label was renamed in the Deployment’s template, but the policy’s selector was not updated.

flowchart TD
    A[Selector mismatch] --> B{Different label?}
    B -->|Yes| C[Fix the selector or the labels]
    B -->|No| D{Missing label?}
    D -->|Yes| C
    D -->|No| E{Namespace mismatch?}
    E -->|Yes| F[Move the policy to the correct namespace]
    E---|No| G{Renamed label?}
    G -->|Yes| C
    G -->|No| H[Unknown]

The remediation

The remediation depends on the cause:

# Substitute your own values before running:
POLICY=billing-allow-ingress
DEPLOY=billing
NS=prod            # namespace the billing Pods actually run in
WRONG_NS=default   # namespace the policy was created in by mistake

# Option 1: Fix the policy's selector
kubectl patch networkpolicy "$POLICY" -n "$NS" -p '{"spec":{"podSelector":{"matchLabels":{"app":"billing"}}}}'

# Option 2: Fix the Pod's labels (via the Deployment)
kubectl patch deployment "$DEPLOY" -n "$NS" -p '{"spec":{"template":{"metadata":{"labels":{"app":"billing"}}}}}'

# Option 3: Move the policy to the correct namespace
kubectl get networkpolicy "$POLICY" -n "$WRONG_NS" -o yaml > policy.yaml
kubectl apply -f policy.yaml -n "$NS"
kubectl delete networkpolicy "$POLICY" -n "$WRONG_NS"

Patching the Deployment’s pod template rolls the Pods; patching the policy takes effect on the next CNI sync without restarting anything.

The Pod’s labels

The Pod’s labels are the source of truth. The policy’s selector must match the Pod’s labels.

# Substitute your own values before running:
POLICY=billing-allow-ingress
NS=prod

# Check the Pod's labels
kubectl get pods -n "$NS" --show-labels

# Output:
# NAME                       READY   STATUS    LABELS
# billing-7d8f-abcde         1/1     Running   app=billing,tier=api
# billing-7d8f-def01         1/1     Running   app=billing,tier=api

# Check the policy's selector
kubectl get networkpolicy "$POLICY" -n "$NS" -o yaml

# Output:
# spec:
#   podSelector:
#     matchLabels:
#       app: billing
#       tier: api

The labels and the selector must match.

Production discipline

A selector mismatch is the cluster’s network hypothesis. The discipline is to walk the canonical flow extended with the network policy, identify the failure mode, apply the remediation. The network is the cluster’s connectivity; the remediation is the selector or the labels.

• Compare the policy’s selector to the Pod’s labels. kubectl get pods --show-labels prints the labels; kubectl get networkpolicy -o yaml prints the selector. No controller reports the mismatch for you.
• Choose the cheaper side of the fix. Patching the Deployment’s pod template rolls the Pods; patching the policy’s selector takes effect on the next CNI sync without restarting anything.

Quiz