Skip to main content
RunBook Academy

KubernetesCX · Priority and PreemptionPriority and preemption

Priority anti-patterns — the most common mistakes

Advanced⏱ ~17 minkubectl

What you'll learn

  • Identify the most common priority anti-patterns
  • Explain why each anti-pattern is a problem
  • Apply the fixes for each anti-pattern
  • Build the operational discipline of using priority deliberately

Prerequisites

Verified against Kubernetes 1.34.x · kubeadm 1.34.x · kubectl 1.34.x · etcd 3.6.x · CoreDNS 1.11.x · containerd 1.7.x / 2.x · 2026-08-16

Not yet marked complete on this device.

Priority anti-patterns are the most common mistakes when using PriorityClass. This lesson walks the anti-patterns, the fixes, and the operational discipline.

Anti-pattern 1: App PriorityClasses using values above 1 billion

# Anti-pattern
apiVersion: scheduling.k8s.io/v1
kind: PriorityClass
metadata:
  name: production
value: 5000000000  # above 1 billion

The intent is “make this really important”; the result is conflict with system classes.

The fix: use values below 1 billion. 1000000 (one million) is more than enough for application workloads.

Anti-pattern 2: No globalDefault

# Anti-pattern: no globalDefault
apiVersion: scheduling.k8s.io/v1
kind: PriorityClass
metadata:
  name: production
value: 1000
# No globalDefault: true

The intent is “explicit priorities are better”; the result is that Pods without explicit priority have priority 0 (the lowest).

The fix: set a sensible globalDefault. A class with value 1000 and globalDefault: true ensures every Pod has a meaningful default.

Anti-pattern 3: Batch workloads with PreemptLowerPriority

# Anti-pattern
apiVersion: scheduling.k8s.io/v1
kind: PriorityClass
metadata:
  name: batch
value: 100
preemptionPolicy: PreemptLowerPriority  # default

The intent is “let batch evict lower-priority Pods”; the result is that batch workloads can evict production.

The fix: set preemptionPolicy: Never for batch workloads. Batch should wait, not evict.

Anti-pattern 4: PDBs with minAvailable: 100%

# Anti-pattern
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: myapp-pdb
spec:
  minAvailable: 100%  # blocks all preemption
  selector:
    matchLabels:
      app: myapp

The intent is “always available”; the result is that no preemption is possible.

The fix: leave headroom. minAvailable: 60-70% of replicas.

Anti-pattern 5: No system classes for critical components

# Anti-pattern: kube-proxy has no priority
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: kube-proxy
  namespace: kube-system
spec:
  template:
    spec:
      # No priorityClassName

The intent is “kube-proxy should always run”; the result is that kube-proxy has priority 0 and can be preempted.

The fix: assign system-node-critical or system-cluster-critical to critical components.

Anti-pattern 6: Priority based on team

# Anti-pattern: priority based on team, not workload
apiVersion: scheduling.k8s.io/v1
kind: PriorityClass
metadata:
  name: team-a-critical
value: 1000000

The intent is “team A is important”; the result is that team A’s workloads always win, regardless of business importance.

The fix: priority is about workload importance, not team politics. Production-critical workloads get high priority; non-critical get lower.

Anti-pattern 7: No monitoring

# Anti-pattern: no alerts on preemption
# (no Prometheus rule)

The intent is “preemption is rare”; the result is silent preemption failures.

The fix: alert on FailedScheduling events, Preempted events, and Pods stuck Pending.

Quiz

Knowledge check · 4 questions

  1. Q1. Why should batch workloads generally not preempt?

  2. Q2. `preemptionPolicy: Never` lets a Pod keep its scheduling priority without evicting others.

  3. Q3. Every team's workload sits at the same high priority while the platform's own components sit below all of them; rebuild the ladder.

    `kubectl get priorityclass` lists six application classes — `team-a-critical`, `team-b-critical`, `team-c-critical` and three more — all at value 1000000, none with `globalDefault`. The monitoring agent DaemonSet and the log shipper carry no `priorityClassName` and were preempted twice last week during a capacity squeeze. Nothing in the cluster distinguishes a checkout Pod from a nightly report.

  4. Q4. What priority does a Pod get when it names no PriorityClass and no class sets `globalDefault: true`, and what does that mean when the cluster is full?

Passing score: 75%. Answers are checked in this browser.

The operational discipline

Priority anti-patterns in production rest on five non-negotiable elements:

  • App classes below 1 billion. Reserved for system classes.
  • Set globalDefault. Pods without explicit priority have a meaningful default.
  • Batch classes with Never. Batch should wait.
  • PDBs with headroom. At least 30% can be unavailable.
  • Critical components get system classes. kube-proxy, CoreDNS, CNI agents.

Priority is the operational signal for what matters. The discipline is to assign priorities deliberately, based on workload importance, and to enforce the pattern consistently.