HA patterns — spreading replicas for availability

High availability in Kubernetes is a combination of replica count, anti-affinity, topology spread, and PodDisruptionBudgets. This lesson walks the canonical HA patterns: replicas across zones for disaster recovery, replicas across nodes for hardware failure, and the combination patterns that give layered protection.

The HA stack

flowchart TB
    A[High availability] --> B[Replicas]
    A --> C["Anti-affinity / topology spread"]
    A --> D[PodDisruptionBudget]
    A --> E[Resource requests]
    B --> F[At least 2 for HA]
    C --> G[Distribute across failure domains]
    D --> H[Bound voluntary disruption]
    E --> I[Reserve capacity for failover]

Every layer is needed. Replicas without distribution produce a single point of failure (multiple Pods on one node). Distribution without replicas produces no failover.

Pattern 1: replicas across zones

spec:
  replicas: 3
  affinity:
    podAntiAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
      - topologyKey: topology.kubernetes.io/zone
        labelSelector:
          matchLabels:
            app: db

Three replicas, one per zone. A zone failure loses one replica; the others continue.

flowchart TB
    subgraph "Zone us-east-1a"
      N1[node-01] --> P1[db-1]
    end
    subgraph "Zone us-east-1b"
      N2[node-02] --> P2[db-2]
    end
    subgraph "Zone us-east-1c"
      N3[node-03] --> P3[db-3]
    end

With replicas: 3 and 3+ zones, each zone gets one replica. With replicas: 6, each zone gets two.

Pattern 2: replicas across nodes

spec:
  replicas: 6
  affinity:
    podAntiAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
      - topologyKey: kubernetes.io/hostname
        labelSelector:
          matchLabels:
            app: web

Six replicas, one per node. A node failure loses one replica; the others continue.

flowchart TB
    N1[node-01] --> P1[web-1]
    N2[node-02] --> P2[web-2]
    N3[node-03] --> P3[web-3]
    N4[node-04] --> P4[web-4]
    N5[node-05] --> P5[web-5]
    N6[node-06] --> P6[web-6]

Pattern 3: layered (zones + nodes)

spec:
  replicas: 6
  affinity:
    podAntiAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
      - topologyKey: kubernetes.io/hostname
        labelSelector:
          matchLabels:
            app: db
      - topologyKey: topology.kubernetes.io/zone
        labelSelector:
          matchLabels:
            app: db

The Pod must not co-locate on the same node (host-level failure) and must not be in the same zone (zone failure). The scheduler satisfies both constraints.

flowchart TB
    subgraph "Zone us-east-1a"
      N1[node-01] --> P1[db-1]
      N2[node-02] --> P2[db-2]
    end
    subgraph "Zone us-east-1b"
      N3[node-03] --> P3[db-3]
      N4[node-04] --> P4[db-4]
    end
    subgraph "Zone us-east-1c"
      N5[node-05] --> P5[db-5]
      N6[node-06] --> P6[db-6]
    end

The result: two replicas per zone, two nodes per zone, no two replicas on the same node. A zone failure loses two replicas; a node failure loses one.

Pattern 4: combined anti-affinity and topology spread

spec:
  replicas: 6
  affinity:
    podAntiAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
      - topologyKey: kubernetes.io/hostname
        labelSelector:
          matchLabels:
            app: db
  topologySpreadConstraints:
  - maxSkew: 1
    topologyKey: topology.kubernetes.io/zone
    whenUnsatisfiable: DoNotSchedule
    labelSelector:
      matchLabels:
        app: db

Anti-affinity enforces node-level separation; topology spread enforces zone-level balance. The combination is “one per node, balanced across zones.”

flowchart TB
    A[Pod arrives] --> B{Anti-affinity:<br/>same node?}
    B -->|yes| C[Eliminate node]
    B -->|no| D["Topology spread:<br/>zone skew?}
    D -->|too high| E[Eliminate zone"]
    D -->|balanced| F[Score phase]
    C --> G[Failed to schedule]
    E --> G

The trade-offs

Strict separation vs flexibility

flowchart LR
    A[Strict anti-affinity] -->|no fallback| B[Pending Pod if insufficient nodes]
    C[Topology spread ScheduleAnyway] -->|falls back| D[Imbalanced distribution]
    E[Preferred anti-affinity] -->|falls back| F[Co-located Pod]

The discipline: choose the failure mode that matches the workload’s availability requirements.

Distribution vs resource headroom

A cluster with strict separation requires more nodes than a cluster with co-located Pods. The HA cost is the node count.

flowchart LR
    A[6 replicas] --> B["Strict separation:<br/>6 nodes needed"]
    A --> C["Co-location:<br/>2-3 nodes sufficient"]
    B --> D[More capacity cost]
    C --> E[Lower capacity cost]

Pattern 5: HA with PodDisruptionBudget

spec:
  replicas: 6
  affinity:
    podAntiAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
      - topologyKey: kubernetes.io/hostname
        labelSelector:
          matchLabels:
            app: web
---
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: web
spec:
  minAvailable: 4
  selector:
    matchLabels:
      app: web

The Deployment spreads across nodes; the PDB guarantees at least 4 Pods are running during voluntary disruptions (drain, eviction). With 6 replicas, at most 2 can be disrupted at any time.

Pattern 6: HA database with quorum

For a database cluster (PostgreSQL, etcd), the quorum requirement is paramount:

flowchart LR
    A[3-node database] --> B{1 node fails}
    B --> C[2 nodes remain]
    C --> D["Quorum: 2 of 3"]
    D --> E[Database continues]
    A2[5-node database] --> B2{2 nodes fail}
    B2 --> C2[3 nodes remain]
    C2 --> D2["Quorum: 3 of 5"]
    D2 --> E2[Database continues]

A 3-replica database can lose 1 node; a 5-replica can lose 2 nodes. The trade-off: cost vs availability.

Anti-patterns

Anti-pattern 1: too many replicas for the cluster

replicas: 100
affinity:
  podAntiAffinity:
    requiredDuringSchedulingIgnoredDuringExecution:
    - topologyKey: kubernetes.io/hostname
      labelSelector:
        matchLabels:
          app: web

A 100-replica Deployment on a 10-node cluster: only 10 Pods scheduled; 90 Pending. The fix: scale the cluster, reduce replicas, or use topology spread.

Anti-pattern 2: HA without resource requests

replicas: 6
containers:
- name: web
  image: web:v1
  # no resources

Without resource requests, the scheduler cannot reserve capacity. A node failure moves the Pod to another node; that node may not have capacity. The fix: set requests.

Anti-pattern 3: HA without PDB

A Deployment with replicas and anti-affinity but no PDB. Voluntary disruption (drain) can take down multiple Pods at once. The fix: add a PDB.

Quiz

Production discipline

• Three replicas minimum for HA. Two replicas protect against single failures but leave no headroom for recovery.
• Distribute across failure domains. Anti-affinity and topology spread are the tools; use them deliberately.
• Combine anti-affinity with topology spread. Anti-affinity enforces strict separation; topology spread enforces balance.
• Set PodDisruptionBudget. HA without a PDB is incomplete; voluntary disruption can defeat it.
• Set resource requests. HA without capacity is theoretical; the cluster must have room for failover.

HA is a stack of patterns, not a single feature. Operators who combine replicas, anti-affinity, topology spread, PDBs, and resource requests have workloads that survive failure.