Multi-worker pools and topology spread — the cluster's compute

A node label does not reserve a pool. Labelling three memory-optimised nodes and adding a nodeSelector to the workload that needs them leaves the pool open to everything else in the cluster, which drifts onto the most expensive nodes available and settles there. Reservation takes a taint, placement takes a selector or affinity, and even distribution takes topology spread constraints — this lesson covers which of the three does what, and what breaks when one is missing.

The multi-worker pools

The multi-worker pools are the cluster’s compute. The pools are:

• General node pool (3 nodes). The standard workloads.
• Memory-optimized node pool (3 nodes). The memory-intensive workloads.
• Compute-optimized node pool (3 nodes). The compute-intensive workloads.

flowchart TB
    subgraph W["Worker Pools (multi-AZ)"]
        W1["General (3 nodes)"]
        W2["Memory-optimized (3 nodes)"]
        W3["Compute-optimized (3 nodes)"]
    end
    W1 --> WORKLOAD_A["Stateless workloads"]
    W2 --> WORKLOAD_B["Memory-intensive workloads"]
    W3 --> WORKLOAD_C["Compute-intensive workloads"]

The multi-worker pools are the cluster’s compute.

The topology spread

The topology spread is the cluster’s distribution. The topology spread constraints are the scheduler’s hints to distribute the Pods across the cluster’s topology.

flowchart TB
    subgraph Z["Zone A"]
        N1["Node 1"]
        N2["Node 2"]
    end
    subgraph Z2["Zone B"]
        N3["Node 3"]
        N4["Node 4"]
    end
    subgraph Z3["Zone C"]
        N5["Node 5"]
        N6["Node 6"]
    end
    POD1["Pod 1"] --> N1
    POD2["Pod 2"] --> N3
    POD3["Pod 3"] --> N5

The topology spread is the cluster’s distribution.

The taints and tolerations

The taints and tolerations are the cluster’s node selection. The taints are applied to the nodes; the tolerations are applied to the Pods.

# Substitute your own values before running:
NODE=ip-192-0-2-11.eu-west-1.compute.internal
DEPLOY=billing-api
NS=production

# Taint a node
kubectl taint nodes "$NODE" workload=memory:NoSchedule

# Tolerate the taint
kubectl patch deployment "$DEPLOY" -n "$NS" -p '{"spec":{"template":{"spec":{"tolerations":[{"key":"workload","operator":"Equal","value":"memory","effect":"NoSchedule"}]}}}}'

The taints and tolerations are the cluster’s node selection.

The deployment

The deployment is the canonical multi-worker deployment:

# Substitute your own values before running:
MEM_NODE=ip-192-0-2-11.eu-west-1.compute.internal
CPU_NODE=ip-192-0-2-21.eu-west-1.compute.internal

# 1. Label the nodes
kubectl label nodes "$MEM_NODE" workload=memory
kubectl label nodes "$CPU_NODE" workload=compute

# 2. Taint the nodes
kubectl taint nodes "$MEM_NODE" workload=memory:NoSchedule
kubectl taint nodes "$CPU_NODE" workload=compute:NoSchedule

# 3. Deploy the workloads with tolerations
kubectl apply -f deployment-with-tolerations.yaml

# 4. Deploy the workloads with topology spread
kubectl apply -f deployment-with-topology-spread.yaml

The deployment is the multi-worker deployment.

The production discipline

The multi-worker pools are the cluster’s hypothesis. The discipline is to walk the 11-step methodology applied to the multi-worker pools, identify the cause, apply the remediation. The cluster’s discipline is the same scale-free: every component is justified.

Quiz

Production discipline

• Walk the 11-step methodology. The methodology is the diagnostic.
• Identify the gaps. The gaps are the cluster’s missing components.
• Deploy the multi-worker pools. The deployment is the cluster’s recovery.
• Verify the topology spread. The verification is the cluster’s evidence.
• Document the topology. The runbook is the cluster’s reference.