Top 20 production anti-patterns — the cluster's hidden failures

The top 20 production anti-patterns are the cluster’s hidden failures. The diagnostic is the anti-patterns, the impact, and the systematic approach to identifying and fixing them. The discipline is the same scale-free: every anti-pattern gets a fix.

The top 20 anti-patterns

The top 20 production anti-patterns are:

1. :latest tags. The image is :latest; the cluster is at the mercy of the registry.
2. No requests. The container declares no CPU or memory requests. The Pod still schedules — that is the problem. The scheduler accounts it as needing nothing, so it fits anywhere, and the node ends up oversubscribed until the kubelet starts evicting under pressure.
3. No limits. The container has no CPU/memory limits; the workload can starve the node.
4. No probes. The container has no readiness/liveness probes; the kubelet cannot detect failures.
5. No PDB. The workload has no PodDisruptionBudget; the workload can be evicted.
6. Privileged containers. The container is privileged; the workload has root access.
7. Default ServiceAccount. The workload uses the default ServiceAccount; the workload has unnecessary privileges.
8. Secrets in manifests. The Secret is in the manifest; the credential is in the git repo.
9. No NetworkPolicies. The workload has no NetworkPolicy; the workload is reachable from any Pod.
10. No backup. The workload has no backup; the data is not recoverable.
11. No etcd test. The etcd snapshot is not tested; the restoration is not rehearsed.
12. Manual edits. The cluster is managed by manual edits; the cluster’s state is not reproducible.
13. No monitoring. The cluster has no monitoring; the failures are not detected.
14. All replicas on one node. The workload’s replicas are on one node; the node failure takes the workload down.
15. No anti-affinity. The workload has no podAntiAffinity; the replicas are on the same node.
16. No topology spread. The workload has no topology spread; the replicas are in the same zone.
17. No resource quota. The namespace has no ResourceQuota; the namespace can starve the cluster.
18. No LimitRange. The namespace has no LimitRange; the namespaces can have unbounded workloads.
19. No Pod Security Standards. The Pod has no pod-security.kubernetes.io/enforce label; the privileged containers are allowed.
20. No RBAC. The user has cluster-admin; the user has unlimited privileges.

flowchart TD
    A[Production anti-patterns] --> B[Image]
    A --> C[Resources]
    A --> D[Probes]
    A --> E[Availability]
    A --> F[Security]
    A --> G[Backup]
    A --> H[Operations]
    A --> I[Multi-tenancy]
    A --> J[Compliance]

The anti-patterns are the cluster’s hidden failures.

The diagnostic

The canonical diagnostic:

# Substitute the path to the manifest you want scanned:
MANIFEST=deploy/checkout-api.yaml

# Use Polaris to detect anti-patterns
polaris audit --format yaml

# Use Datree to detect anti-patterns
datree test "$MANIFEST"

# Use kube-score to detect anti-patterns
kube-score score "$MANIFEST"

The diagnostic is the anti-pattern detection.

The high-impact anti-patterns

The high-impact anti-patterns are the ones that cause the most production failures:

• :latest tags. The cluster upgrades silently.
• No requests. The scheduler treats the Pod as free and overcommits the node.
• No limits. The workload starves the node.
• No probes. The kubelet cannot detect failures.
• No PDB. The workload is evicted.
• Privileged containers. The workload has root access.
• Default ServiceAccount. The workload has unnecessary privileges.
• No backup. The data is not recoverable.
• No etcd test. The restoration is not rehearsed.
• All replicas on one node. The workload is on a single point of failure.

The high-impact anti-patterns are the production’s most dangerous.

The remediation

The remediation depends on the anti-pattern:

# Option 1: Use a Polaris pipeline
# (CI pipeline that detects anti-patterns)

# Option 2: Use a Datree policy
# (GitOps policy that detects anti-patterns)

# Option 3: Use a kube-score pipeline
# (CI pipeline that scores the manifests)

The remediation is the anti-pattern fix.

Production discipline

Anti-patterns are the cluster’s hypothesis. The discipline is to walk the 11-step methodology applied to the anti-patterns, identify the cause, apply the remediation. The cluster’s discipline is the same scale-free: every anti-pattern gets a fix.

• Scan the manifests, then verify the behaviour. Polaris, kube-score, and Datree score the spec you submitted; a PDB whose selector matches no Pods and three replicas on one node both score as a pass.
• Prioritise by blast radius. :latest tags, missing requests and limits, missing probes, missing PDBs, and privileged containers are the anti-patterns that cause the most production failures.
• Run the detection in CI. A scan that runs only when someone remembers is not prevention; the pipeline is where the anti-pattern is caught before it reaches the cluster.

Quiz