Topology affinity — the binding constraints between zones, regions, and nodes

Topology affinity is the constraint that ties a PVC to a zone, a region, or a node. Misconfigured topology is a common cause of Pending PVCs in multi-AZ clusters. This lesson walks the constraint, the diagnostic, and the production fix.

Topology in Kubernetes

Kubernetes topology is expressed via node labels:

kubectl get nodes --show-labels | grep topology
# node-1   ...   topology.kubernetes.io/zone=us-east-1a
#                    topology.kubernetes.io/region=us-east-1
# node-2   ...   topology.kubernetes.io/zone=us-east-1b
#                    topology.kubernetes.io/region=us-east-1
# node-3   ...   topology.kubernetes.io/zone=us-east-1c
#                    topology.kubernetes.io/region=us-east-1

The standard labels:

• topology.kubernetes.io/zone: the AZ.
• topology.kubernetes.io/region: the region.
• kubernetes.io/hostname: the node name (older).

A StorageClass can constrain the topology where PVs are created:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: db-ssd
provisioner: ebs.csi.aws.com
parameters:
  type: io2
volumeBindingMode: WaitForFirstConsumer
allowedTopologies:
- matchLabelExpressions:
  - key: topology.kubernetes.io/zone
    values: ["us-east-1a", "us-east-1b", "us-east-1c"]
- matchLabelExpressions:
  - key: topology.kubernetes.io/region
    values: ["us-east-1"]

The binding logic

When a PVC is submitted with a StorageClass that has allowedTopologies:

1. The PVC is created; binding is deferred.
2. A Pod that consumes the PVC is scheduled to a node.
3. The scheduler checks the node’s topology labels against the StorageClass’s allowedTopologies.
4. If the node matches, the PV is created in the matching topology.
5. If the node does not match, the Pod cannot be scheduled; the PVC remains Pending.

sequenceDiagram
    participant U as User
    participant API as API server
    participant S as Scheduler
    participant CSI as CSI controller
    U->>API: submit PVC
    API-->>U: PVC Pending
    U->>API: create Pod
    API->>S: schedule Pod
    S->>S: check node topology vs allowedTopologies
    alt node matches
        S->>CSI: CreateVolume in node's zone
        CSI-->>API: PV created
        API-->>U: PVC bound
    else node does not match
        S-->>API: Pod cannot schedule
        API-->>U: Pod Pending
    end

Cause 1: missing allowedTopologies

A multi-AZ cluster with WaitForFirstConsumer but no allowedTopologies. The PV can be created in any zone, including zones that the Pod cannot reach. The PVC may be Pending if the cluster has nodes in zones outside the StorageClass’s reach.

# Substitute your own value before running:
PVC=data-postgres-0

kubectl describe pvc "$PVC"
# Events:
#   Warning  ProvisioningFailed  ... no topology constraints found

The fix: add allowedTopologies to the StorageClass.

Cause 2: AZ mismatch

The StorageClass’s allowedTopologies does not include the Pod’s AZ:

# StorageClass allows us-east-1a and us-east-1b
# Pod is scheduled to us-east-1c

The Pod cannot be scheduled to the matching zone; the PVC remains Pending.

The fix: add the missing zone to allowedTopologies, or remove the constraint.

Cause 3: regional mismatch

The StorageClass’s allowedTopologies is for a different region:

# StorageClass allows us-east-1
# Cluster has nodes in eu-west-1

The Pods cannot be scheduled; the PVC remains Pending.

The fix: align the StorageClass’s allowedTopologies with the cluster’s region, or create a separate StorageClass for each region.

The diagnostic ladder

flowchart TD
    A[PVC Pending: topology] --> B[kubectl describe pvc]
    B --> C{Topology event?}
    C -->|no constraints| D[Add allowedTopologies]
    C -->|AZ mismatch| E[Fix allowedTopologies]
    C -->|regional mismatch| F[Fix region]
    C -->|node labels missing| G[Add topology labels]

# Substitute your own values before running:
SC=db-ssd
PVC=data-postgres-0
POD=postgres-0

# Step 1: Check the cluster's topology labels
kubectl get nodes --show-labels | grep topology

# Step 2: Check the StorageClass's allowedTopologies
kubectl get storageclass "$SC" -o yaml

# Step 3: Check the PVC's events
kubectl describe pvc "$PVC"

# Step 4: Check the Pod's events (if scheduled)
kubectl describe pod "$POD"

The production fix

For each cause:

The standard production StorageClass for a multi-AZ cluster:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: db-ssd
provisioner: ebs.csi.aws.com
parameters:
  type: io2
  iops: "10000"
reclaimPolicy: Retain
allowVolumeExpansion: true
volumeBindingMode: WaitForFirstConsumer
allowedTopologies:
- matchLabelExpressions:
  - key: topology.kubernetes.io/zone
    values: ["us-east-1a", "us-east-1b", "us-east-1c"]
- matchLabelExpressions:
  - key: topology.kubernetes.io/region
    values: ["us-east-1"]
mountOptions:
- debug

The StorageClass is explicit about which zones and which region are valid; PVCs that request this StorageClass will bind correctly.

Quiz

Production discipline

• allowedTopologies + WaitForFirstConsumer = topology correctness. Use both for multi-AZ clusters.
• Verify the cluster’s topology labels. The labels must match allowedTopologies.
• Document the StorageClass’s topology constraints. Each StorageClass has a topology region and zone list.
• Test topology Pending. A regular validation that submits a PVC to a node in a different zone and verifies the Pending state.