Availability anti-patterns — the workload's resilience

The availability anti-patterns are the workload’s resilience. The diagnostic is the anti-patterns, the impact, and the systematic approach to fixing them. The discipline is the same scale-free: every anti-pattern gets a fix.

The availability anti-patterns

The availability anti-patterns are:

• No PDB. The workload has no PodDisruptionBudget; the workload can be evicted.
• All replicas on one node. The workload’s replicas are on one node; the node failure takes the workload down.
• No anti-affinity. The workload has no podAntiAffinity; the replicas are on the same node.
• No topology spread. The workload has no topology spread; the replicas are in the same zone.

flowchart TD
    A[Availability anti-patterns] --> B[No PDB]
    A --> C[All replicas on one node]
    A --> D[No anti-affinity]
    A --> E[No topology spread]

The availability anti-patterns are the workload’s resilience.

The diagnostic

The canonical diagnostic:

# Substitute your own value before running:
NS=production

# Use Polaris to detect anti-patterns
polaris audit --format yaml

# Check the workload's PDB
kubectl get pdb -n "$NS"

# Check the workload's replicas
kubectl get pods -n "$NS" -o wide

# Check the workload's topology
kubectl get pods -n "$NS" -o jsonpath='{range .items[*]}{.metadata.name}{"\t"}{.spec.nodeName}{"\n"}{end}'

The diagnostic is the Polaris audit, the workload’s PDB, the workload’s replicas, and the workload’s topology.

The remediation

The remediation depends on the anti-pattern:

# Substitute your own values before running. APP is the value of the `app`
# label the Deployment puts on its pods, which the selectors below match:
DEPLOY=checkout-api
NS=production
APP=checkout-api

# Option 1: Set the PDB
kubectl apply -f pdb.yaml

# Option 2: Set the anti-affinity
kubectl patch deployment "$DEPLOY" -n "$NS" -p '{"spec":{"template":{"spec":{"affinity":{"podAntiAffinity":{"requiredDuringSchedulingIgnoredDuringExecution":[{"labelSelector":{"matchLabels":{"app":"'"$APP"'"}},"topologyKey":"kubernetes.io/hostname"}}]}}}}}}'

# Option 3: Set the topology spread
kubectl patch deployment "$DEPLOY" -n "$NS" -p '{"spec":{"template":{"spec":{"topologySpreadConstraints":[{"maxSkew":1,"topologyKey":"topology.kubernetes.io/zone","whenUnsatisfiable":"DoNotSchedule","labelSelector":{"matchLabels":{"app":"'"$APP"'"}}}]}}}}'

The remediation is the anti-pattern fix.

Production discipline

The availability anti-patterns are the cluster’s hypothesis. The discipline is to walk the 11-step methodology applied to the anti-patterns, identify the cause, apply the remediation. The cluster’s discipline is the same scale-free: every anti-pattern gets a fix.

• Run the detection in CI. The CI is the cluster’s prevention.

Quiz