requiredDuringSchedulingIgnoredDuringExecution — hard inter-pod anti-affinity

requiredDuringSchedulingIgnoredDuringExecution for Pod anti-affinity is the canonical pattern for spreading Deployment replicas across nodes: the new Pod must not land on a node already hosting a Pod with the same selector. This lesson covers the standard variant, the common spreading patterns, and when to reach for topology spread instead.

The standard pattern

spec:
  affinity:
    podAntiAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
      - topologyKey: kubernetes.io/hostname
        labelSelector:
          matchLabels:
            app: web

The Pod must not land on a node already hosting a Pod labelled app: web. With replicas: 6 and 6+ nodes, each Pod lands on a different node.

flowchart LR
    A[web Pod] -->|anti-affinity:<br/>app=web| B[Node with web]
    A --> C[Node without web]
    B -.->|rejected| A
    C --> D[Scheduled here]

Why “Ignored at execution”

The standard variant ignores the constraint at execution time. If a node’s labels change after scheduling (e.g., a new Pod labelled app: web is created), the existing Pod is not evicted. The anti-affinity is checked only at scheduling.

flowchart TB
    A["Pod scheduled on node-01<br/>no other web Pod"] --> B["Another web Pod scheduled<br/>on node-01 by mistake"]
    B --> C{Standard variant}
    C -->|IgnoredDuringExecution| D[Existing Pod not evicted]
    C -->|RequiredDuringExecution| E[Existing Pod evicted]

The standard variant is the common choice because eviction on every label change is destructive. The strict variant is reserved for hard correctness.

The canonical use case: spread replicas

spec:
  replicas: 6
  selector:
    matchLabels:
      app: web
  template:
    metadata:
      labels:
        app: web
    spec:
      affinity:
        podAntiAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
          - topologyKey: kubernetes.io/hostname
            labelSelector:
              matchLabels:
                app: web
      containers:
      - name: web
        image: web:v1

The Deployment’s Pods spread across nodes. Every replica needs a node holding no app: web Pod, so the schedulable replica count is capped by the node count. With fewer than 6 nodes, the surplus Pods stay Pending and the scheduler reports didn't match pod anti-affinity rules.

flowchart TB
    subgraph "6 nodes, 6 replicas"
      N1[node-01] --> P1[web-1]
      N2[node-02] --> P2[web-2]
      N3[node-03] --> P3[web-3]
      N4[node-04] --> P4[web-4]
      N5[node-05] --> P5[web-5]
      N6[node-06] --> P6[web-6]
    end

Spread across zones

spec:
  affinity:
    podAntiAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
      - topologyKey: topology.kubernetes.io/zone
        labelSelector:
          matchLabels:
            app: db

Spread across zones. A zone failure loses the replicas in that zone; replicas in other zones continue.

flowchart TB
    subgraph "Zone us-east-1a"
      N1[node-01] --> P1[db-1]
    end
    subgraph "Zone us-east-1b"
      N2[node-02] --> P2[db-2]
    end
    subgraph "Zone us-east-1c"
      N3[node-03] --> P3[db-3]
    end

The cost

Strict separation is expensive for the scheduler:

flowchart LR
    A[Scheduler receives Pod] --> B[Filter phase]
    B --> C[Anti-affinity check]
    C --> D{For each node...}
    D --> E{For each Pod on node...}
    E --> F{Selector matches?}
    F --> G[Node rejected if match]
    D --> H[Walk all nodes, all Pods]
    H --> I["Cache amortises,<br/>but startup is slow"]

For a cluster with thousands of Pods per node and a selector matching the Pod’s own Deployment, every scheduling decision walks many Pods. The cost is paid at scheduling time; the scheduler caches the results.

For Deployments with hundreds of replicas, the cost becomes significant. Topology spread (Part XXV) is the better choice for large-scale spreading.

Failure mode: not enough nodes

flowchart TB
    A["Deployment with<br/>replicas: 10"] --> B["Anti-affinity:<br/>spread across nodes"]
    B --> C{Cluster has<br/>10+ nodes?}
    C -->|no| D["Pod Pending:<br/>cannot satisfy"]
    C -->|yes| E[All Pods scheduled]

With replicas: 10 and a cluster of 6 nodes, the Deployment cannot fully spread. Some Pods sit Pending.

Required anti-affinity vs topology spread

flowchart TB
    A{Strict or balanced?}
    A -->|Strict: must not co-locate| B[Required anti-affinity]
    A -->|Balanced: prefer even distribution| C[Topology spread]
    B --> D["Failure: Pending Pod"]
    C --> E["Failure: tolerated skew"]

For most “spread replicas” use cases, topology spread is the better tool. Required anti-affinity is reserved for hard separation (e.g., a primary and replica that must not co-locate).

Production patterns

Pattern 1: spread web replicas

affinity:
  podAntiAffinity:
    requiredDuringSchedulingIgnoredDuringExecution:
    - topologyKey: kubernetes.io/hostname
      labelSelector:
        matchLabels:
          app: web

Standard pattern. With sufficient nodes, each web Pod is on a different node.

Pattern 2: spread database replicas

affinity:
  podAntiAffinity:
    requiredDuringSchedulingIgnoredDuringExecution:
    - topologyKey: topology.kubernetes.io/zone
      labelSelector:
        matchLabels:
          app: db

Spread across zones. A zone failure loses one replica.

Pattern 3: combine with affinity

affinity:
  podAntiAffinity:
    requiredDuringSchedulingIgnoredDuringExecution:
    - topologyKey: kubernetes.io/hostname
      labelSelector:
        matchLabels:
          app: web
  podAffinity:
    preferredDuringSchedulingIgnoredDuringExecution:
    - weight: 80
      podAffinityTerm:
        topologyKey: kubernetes.io/hostname
        labelSelector:
          matchLabels:
            app: cache

Spread web replicas; prefer co-location with cache.

Common mistakes

Mistake 1: too strict separation

# A 10-replica Deployment on a 5-node cluster
replicas: 10
affinity:
  podAntiAffinity:
    requiredDuringSchedulingIgnoredDuringExecution:
    - topologyKey: kubernetes.io/hostname
      labelSelector:
        matchLabels:
          app: web

5 Pods scheduled; 5 Pending. The fix: topology spread or preferred anti-affinity.

Mistake 2: anti-affinity on StatefulSets

A StatefulSet’s Pods already have stable ordinals; the scheduler spreads them automatically. Adding anti-affinity is redundant and can break the StatefulSet.

Mistake 3: anti-affinity with broad selector

labelSelector:
  matchLabels:
    app: any-app   # matches many Pods

A selector that matches many Pods requires walking many Pods. Performance degrades.

Quiz

Production discipline

• Default to topology spread for replica distribution. Anti-affinity is for hard separation; topology spread is for balanced distribution.
• Verify the cluster’s node count. Required anti-affinity with insufficient nodes produces Pending Pods.
• Combine with affinity for co-location. Spread replicas; co-locate with their dependencies.
• Audit the resulting distribution. A dashboard that shows per-node Pod counts catches distribution failures.
• Use IgnoredDuringExecution. The standard variant is the right default; strict variant is for hard correctness.

Required Pod anti-affinity is the right tool for strict separation. Operators who use it deliberately have workloads that distribute predictably.