Over-permissioned ServiceAccounts — the most common RBAC failure

Over-permissioned ServiceAccounts are the most common RBAC failure in production Kubernetes. A SA with cluster-admin is one compromised Pod away from a full cluster compromise. A SA with * verbs on Pods is one compromised Pod away from a workload takeover. This lesson covers the patterns, the audit, and the migration.

The most common patterns

Five patterns recur in production:

1. Helm chart defaults bind to cluster-admin. Many Helm charts ship with a ClusterRoleBinding to cluster-admin for the chart’s SA. The intent is to make the install “just work.” The result is that every install of that chart creates a cluster-admin SA.

# From a typical Helm chart's rbac.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: {{ .Chart.Name }}
subjects:
- kind: ServiceAccount
  name: {{ .Chart.Name }}
  namespace: {{ .Release.Namespace }}
roleRef:
  kind: ClusterRole
  name: cluster-admin
  apiGroup: rbac.authorization.k8s.io

2. Wildcard ClusterRole. A chart’s ClusterRole has apiGroups: ['*'], resources: ['*'], verbs: ['*']. The SA bound to this ClusterRole can do anything.
3. Default SA with broad binding. A namespace’s default SA is bound to a Role or ClusterRole that allows write operations. Every Pod that does not explicitly use a different SA inherits this.
4. Operator SA with cluster-wide write. A monitoring or logging operator SA is bound to a ClusterRole that allows create, update, patch, delete on resources it does not manage. The operator can modify RBAC objects, CustomResourceDefinitions, and admission policies.
5. CI/CD SA with no minimum-surface discipline. The CI SA is bound to a Role with all verbs on Deployments and Services, plus read on Secrets. A compromise of the CI runner yields the same.

Auditing over-permissioned SAs

The audit walks every SA and compares its effective permissions to the expected surface:

#!/bin/bash
# audit-sas.sh — find over-permissioned ServiceAccounts

declare -A EXPECTED=(
  ["ci:ci-runner"]="deployments [get, list, watch]"
  ["monitoring:prometheus"]="nodes/metrics [] services [get, list, watch]"
  ["prod:api"]="configmaps [get] pods [get, list, watch]"
)

for ns in $(kubectl get ns -o name | cut -d/ -f2); do
  for sa in $(kubectl get sa -n "$ns" -o name | cut -d/ -f2); do
    KEY="$ns:$sa"
    EFFECTIVE=$(kubectl auth can-i --list -n "$ns" \
      --as=system:serviceaccount:$ns:$sa 2>/dev/null)
    
    # Check for the worst patterns
    if kubectl auth can-i create clusterrolebindings \
       --as=system:serviceaccount:$ns:$sa 2>/dev/null; then
      echo "CRITICAL: $KEY can create ClusterRoleBindings"
    fi
    if kubectl auth can-i '*' '*' \
       --as=system:serviceaccount:$ns:$sa 2>/dev/null; then
      echo "CRITICAL: $KEY has wildcard RBAC"
    fi
  done
done

The script reports SAs that can create RBAC objects (privilege escalation primitives) and SAs with wildcard RBAC. Both are Critical findings.

Verifying with kubectl auth can-i

For a single SA, the verification is:

# Effective permissions
kubectl auth can-i --list -n prod \
  --as=system:serviceaccount:prod:api

# Worst-case: can it create RBAC objects?
kubectl auth can-i create rolebindings -n prod \
  --as=system:serviceaccount:prod:api
# Should be: no

# Worst-case: can it read secrets?
kubectl auth can-i get secrets -n prod \
  --as=system:serviceaccount:prod:api
# Should be: no (unless the workload genuinely needs secrets)

Migrating to minimum-surface SAs

The migration has three steps:

1. Audit every SA’s effective permissions and compare to the expected surface.
2. Document the minimum surface for each SA (the verbs × resources the workload needs).
3. Replace the over-permissioned binding with a minimum-surface binding.

# Before: cluster-admin
kind: ClusterRoleBinding
subjects:
- kind: ServiceAccount
  name: myapp
  namespace: prod
roleRef:
  kind: ClusterRole
  name: cluster-admin

# After: minimum surface
kind: Role
metadata:
  name: myapp-runtime
  namespace: prod
rules:
- apiGroups: [""]
  resources: ["configmaps"]
  verbs: ["get"]
  resourceNames: ["app-config"]
---
kind: RoleBinding
metadata:
  name: myapp-runtime
  namespace: prod
subjects:
- kind: ServiceAccount
  name: myapp
  namespace: prod
roleRef:
  kind: Role
  name: myapp-runtime

The new Role allows the workload to read one specific ConfigMap. The old cluster-admin binding is removed.

Production failure modes

1. Helm chart reinstalls the cluster-admin binding. The chart was fixed; the cluster was upgraded; the next chart install re-creates the binding. The fix is to track the chart’s RBAC and re-audit after every install.
2. Default SA in a namespace has broad RBAC. A developer deployed a workload with no SA specified; it uses default; default is bound to cluster-admin. The workload is now cluster-admin.
3. Audit script is not run after every Helm install. The audit runs quarterly; the chart installs weekly. The audit always finds stale data. The fix is to run the audit in CI/CD against every Helm install.
4. CI/CD SA is over-permissioned to “make things work.” The CI SA has cluster-admin because the team did not want to debug RBAC failures. The fix is to write minimum-surface Roles and update the CI SA’s bindings.

Cross-course references

• The Observability course covers the audit log entries for over-permissioned SA usage.
• The Linux course covers the file permissions on Secret data that an over-permissioned SA can read.

Quiz

Production discipline

Over-permissioned SAs are the most common RBAC failure because Helm charts, tutorials, and “just make it work” defaults ship with cluster-admin bindings. The discipline is to audit every SA’s effective permissions, document the minimum surface for each, and replace the over-permissioned bindings with minimum-surface bindings. The audit runs in CI/CD against every Helm install, and a quarterly sweep catches the drift. A cluster whose SAs are over-permissioned has an RBAC programme that is not defensible; a cluster whose SAs are minimum-surface has an RBAC programme that is auditable.