PDB best practices — production discipline for protected workloads

The PDB is the cluster’s contract with the workload. The standard patterns are: HTTP services with minAvailable, stateful workloads with maxUnavailable, and queue consumers with minAvailable. This lesson walks the patterns, the interaction with rolling updates, and the operational discipline.

The PDB’s three standard patterns

The three standard patterns for PDBs:

flowchart TD
    A[Workload type] --> B[HTTP service]
    A --> C[Stateful workload]
    A --> D[Queue consumer]
    B --> E[minAvailable: replica_count - 1]
    C --> F[maxUnavailable: 1]
    D --> G[minAvailable: consumer_count - 1]

The patterns are the cluster’s protection against the drain. The patterns match the workload’s redundancy model.

Pattern 1: HTTP service with minAvailable

A stateless HTTP service with N replicas:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: billing
spec:
  replicas: 3
  selector:
    matchLabels:
      app: billing
  template:
    metadata:
      labels:
        app: billing
    spec:
      containers:
        - name: billing
          image: registry.example.com/billing:1.0.0
---
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: billing-pdb
spec:
  minAvailable: 2
  selector:
    matchLabels:
      app: billing

The PDB’s minAvailable: 2 allows 1 Pod to be evicted. The drain is allowed; the workload is protected.

The pattern is the standard for HTTP services. The service has 3 replicas; the PDB allows 1 to be evicted; the service is still serving.

Pattern 2: Stateful workload with maxUnavailable

A StatefulSet with N replicas:

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: database
spec:
  replicas: 3
  serviceName: database
  selector:
    matchLabels:
      app: database
  template:
    metadata:
      labels:
        app: database
    spec:
      containers:
        - name: database
          image: registry.example.com/database:1.0.0
---
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: database-pdb
spec:
  maxUnavailable: 1
  selector:
    matchLabels:
      app: database

The PDB’s maxUnavailable: 1 allows 1 Pod to be unavailable. A StatefulSet already replaces its Pods one ordinal at a time, and the PDB holds the drain to that same rate.

The pattern is the standard for stateful workloads. The stateful workload has 3 replicas; the PDB allows 1 to be unavailable; the workload is still serving.

Pattern 3: Queue consumer with minAvailable

A queue consumer with N replicas:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: consumer
spec:
  replicas: 3
  selector:
    matchLabels:
      app: consumer
  template:
    metadata:
      labels:
        app: consumer
    spec:
      containers:
        - name: consumer
          image: registry.example.com/consumer:1.0.0
---
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: consumer-pdb
spec:
  minAvailable: 2
  selector:
    matchLabels:
      app: consumer

The PDB’s minAvailable: 2 allows 1 Pod to be evicted. Two consumers keep draining the queue while the third is rescheduled.

The pattern is the standard for queue consumers. The consumer has 3 replicas; the PDB allows 1 to be evicted; the queue is still being processed.

The PDB’s interaction with the rolling update

The PDB’s interaction with the rolling update:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: billing
spec:
  replicas: 3
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 1
      maxSurge: 1
  selector:
    matchLabels:
      app: billing
  template:
    metadata:
      labels:
        app: billing
    spec:
      containers:
        - name: billing
          image: registry.example.com/billing:1.0.0
---
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: billing-pdb
spec:
  minAvailable: 2
  selector:
    matchLabels:
      app: billing

The rolling update’s maxUnavailable: 1 and the PDB’s minAvailable: 2 are consistent. The rolling update evicts 1 Pod; the PDB allows the eviction; the rolling update continues.

The PDB’s minAvailable and the rolling update’s maxUnavailable must be consistent. A PDB with minAvailable: 3 and a rolling update with maxUnavailable: 1 is a configuration that works. A PDB with minAvailable: 3 and a rolling update with maxUnavailable: 0 is a configuration that fails.

The PDB’s percentage values

The PDB’s percentage values are the cluster’s protection against the workload’s scale:

spec:
  minAvailable: "50%"

The percentage is computed against the PDB’s expectedPods. The PDB’s expectedPods is the total number of Pods that the PDB applies to.

The percentage is the cluster’s protection against the workload’s scale. A Deployment with 5 replicas and minAvailable: 50% has minAvailable: 3. A Deployment with 10 replicas and minAvailable: 50% has minAvailable: 5.

The PDB’s management at scale

The PDB’s management at scale:

kubectl get pdb -A

The output shows the PDB’s status. The operator should monitor the output for unhealthy PDBs.

The PDB’s metrics:

promtool query instant http://prometheus:9090 \
  'kube_poddisruptionbudget_status'

The metric returns the PDB’s status. The operator should monitor the metric and alert on the threshold.

The PDB’s automation

The PDB’s automation is the cluster’s mechanism for managing the PDBs at scale. The automation:

• Creates the PDBs. The automation creates the PDBs for every production-critical workload.
• Validates the PDBs. The automation validates the PDBs against the workload’s spec.
• Updates the PDBs. The automation updates the PDBs when the workload’s spec changes.
• Removes the PDBs. The automation removes the PDBs when the workload is deleted.

The PDB’s automation is the cluster’s protection against misconfigured PDBs.

The PDB’s audit

The PDB’s audit:

kubectl get pdb -A -o yaml

The output shows the PDB’s spec. The audit checks:

• The PDB’s selector matches the workload’s Pods.
• The PDB’s minAvailable or maxUnavailable is consistent with the workload’s replica count.
• The PDB’s currentHealthy is greater than the desiredHealthy.

The audit is the cluster’s protection against the PDB’s anti-patterns.

The PDB’s monitoring

The PDB’s monitoring:

groups:
  - name: pdb
    rules:
      - alert: PDBDisruptionsNotAllowed
        expr: kube_poddisruptionbudget_status{condition="Disallowed"} > 0
        for: 5m
        labels:
          severity: warning
        annotations:
          summary: "PDB {{ $labels.poddisruptionbudget }} is not allowing disruptions"
          description: "PDB {{ $labels.poddisruptionbudget }} has disruptionsAllowed = 0 for more than 5 minutes."

The alert fires when the PDB’s disruptionsAllowed is 0 for more than 5 minutes. The alert is the operator’s signal that the PDB is blocking the drain.

Quiz

Production discipline

• The PDB is the cluster’s contract with the workload. The standard patterns are: HTTP service with minAvailable, stateful with maxUnavailable, queue consumer with minAvailable.
• Set the PDB’s value to allow the drain. The minAvailable should be one less than the replica count; the maxUnavailable should be one.
• Use percentage values for elastic workloads. A Deployment with variable replica count should use the percentage value.
• Match the PDB to the rolling update strategy. The PDB’s value must be consistent with the rolling update’s maxUnavailable.
• Audit the PDB at every release. The PDB’s configuration should be version-controlled; the audit catches the failures.
• Monitor the PDB’s status. The PDB’s disruptionsAllowed is the operator’s primary signal.
• Test the PDB in non-production. A staging cluster that mirrors production is the right place to test the PDB.
• Document the PDB’s intent. A PDB that does not have a documented intent is a PDB that does not protect the workload.