requiredDuringSchedulingIgnoredDuringExecution — hard node affinity

Node Affinity with requiredDuringSchedulingIgnoredDuringExecution is the hard-constraint node-placement mechanism. The Pod must land on a node that satisfies the rule at scheduling time; the rule is then ignored (the Pod stays even if the node’s labels change later). This lesson covers the syntax, the set-based operators, the composition rules, and the production failure modes.

The syntax

spec:
  affinity:
    nodeAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        nodeSelectorTerms:
        - matchExpressions:
          - key: topology.kubernetes.io/zone
            operator: In
            values: ["us-east-1a", "us-east-1b"]
        - matchExpressions:
          - key: kubernetes.io/arch
            operator: In
            values: ["amd64"]

The structure:

• nodeSelectorTerms — list of terms; OR composition.
• matchExpressions (or matchFields) — list of expressions within a term; AND composition.

flowchart TB
    A[nodeSelectorTerms] -->|OR| B[Term 1]
    A -->|OR| C[Term 2]
    B -->|AND| D[Expression 1]
    B -->|AND| E[Expression 2]
    C -->|AND| F[Expression 1]

A node matches the rule if it satisfies any term. A term is satisfied if all its matchExpressions are true.

Set-based operators

Node Affinity supports set-based operators that nodeSelector does not:

flowchart LR
    A[Match expressions] --> B["key=disk<br/>operator=In<br/>values=ssd,nvme"]
    B --> C{"disk in<br/>[ssd, nvme]?"}
    C -->|yes| D[Match]
    C -->|no| E[No match]

Composition: OR at the term level

nodeSelectorTerms:
- matchExpressions:
  - key: topology.kubernetes.io/zone
    operator: In
    values: ["us-east-1a"]
- matchExpressions:
  - key: topology.kubernetes.io/zone
    operator: In
    values: ["us-east-1b"]

A node matches if zone in [us-east-1a] OR zone in [us-east-1b]. This is “zone is us-east-1a or us-east-1b” — the OR is at the term level.

Composition: AND at the expression level

nodeSelectorTerms:
- matchExpressions:
  - key: topology.kubernetes.io/zone
    operator: In
    values: ["us-east-1a"]
  - key: kubernetes.io/arch
    operator: In
    values: ["amd64"]

A node matches if zone in [us-east-1a] AND arch in [amd64]. The AND is at the expression level.

Ignored at execution time

The IgnoredDuringExecution part means: after the Pod is scheduled, the rule is ignored. If the node’s labels change (or the node’s label set is altered), the Pod continues to run.

flowchart TB
    A[Pod scheduled on node-01] --> B{node-01 labels<br/>change later?}
    B -->|yes| C["Pod stays on node-01<br/>rule ignored"]
    B -->|no| D[Pod continues]

This is the standard behaviour. The RequiredDuringExecution variant (rare) would evict the Pod if the node’s labels change.

Production patterns

Pattern 1: zone pinning with failover

nodeAffinity:
  requiredDuringSchedulingIgnoredDuringExecution:
    nodeSelectorTerms:
    - matchExpressions:
      - key: topology.kubernetes.io/zone
        operator: In
        values: ["us-east-1a", "us-east-1b"]

The Pod must land in one of two zones. If both zones fail, the Pod is unschedulable.

Pattern 2: arch + disk

nodeAffinity:
  requiredDuringSchedulingIgnoredDuringExecution:
    nodeSelectorTerms:
    - matchExpressions:
      - key: kubernetes.io/arch
        operator: In
        values: ["amd64"]
      - key: disk
        operator: In
        values: ["ssd"]

The Pod must land on an amd64 node with SSD. Both must be true.

Pattern 3: exclusion

nodeAffinity:
  requiredDuringSchedulingIgnoredDuringExecution:
    nodeSelectorTerms:
    - matchExpressions:
      - key: experimental
        operator: DoesNotExist

The Pod must not land on a node with the experimental label. Useful for separating workloads from canary nodes.

Pattern 4: combination with nodeSelector

spec:
  nodeSelector:
    disk: ssd
  affinity:
    nodeAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        nodeSelectorTerms:
        - matchExpressions:
          - key: topology.kubernetes.io/zone
            operator: In
            values: ["us-east-1a", "us-east-1b"]

The Pod must match both: nodeSelector (AND) AND Node Affinity. The combination is ANDed.

Failure modes

Failure 1: No nodes match

flowchart TB
    A[Pod Pending] --> B[Filter phase]
    B --> C{Node Affinity match?}
    C -->|no| D[Eliminated]
    D --> E[All nodes eliminated]
    E --> F["FailedScheduling:<br/>0/N nodes didn't match<br/>Pod's node affinity"]

The fix: correct the rule, add nodes with the labels, or relax the constraint.

Failure 2: typo in the operator

matchExpressions:
- key: zone
  operator: IsIn  # typo; should be "In"

The API server rejects the Pod (unknown operator). Fix the typo.

Failure 3: empty values list

matchExpressions:
- key: zone
  operator: In
  values: []   # empty

An empty values list with In matches no node. The Pod is unschedulable.

Failure 4: integer comparison without numeric value

matchExpressions:
- key: priority
  operator: Gt
  values: ["high"]   # not numeric

Gt and Lt parse the value as a number. "high" is not numeric; the comparison fails. Use In or NotIn for non-numeric labels.

Inspection

kubectl describe pod web-7c8d9b1f8-abcd -n prod | grep -A 5 Node-Selectors
# Node-Selectors:
#   topology.kubernetes.io/zone In [us-east-1a us-east-1b]
#   kubernetes.io/arch In [amd64]

The scheduler renders the rule in human-readable form. Use it to verify the rule is what you intended.

Quiz

Production discipline

• Required affinity is hard. If no node matches, the Pod is unschedulable. Use preferred affinity for soft constraints.
• Compose OR at the term level, AND at the expression level. Multiple terms are ORed; multiple expressions within a term are ANDed.
• Validate the rule in CI. A typo or empty values list is a Pending Pod waiting to happen.
• Combine with nodeSelector. A nodeSelector and a Node Affinity are ANDed; both must be satisfied.
• Document the failure recovery. A Pod that requires a zone with no nodes is unschedulable; the recovery is to add nodes, not to weaken the rule.

Required Node Affinity is the right tool for hard placement constraints. Operators who understand the composition rules write rules that work; operators who do not write Pending Pods.