Controller cluster permissions — what the reconciler is allowed to do

Every GitOps control discussed so far is a decision made by software reading configuration. The controller decides whether a source is allowed; the controller decides whether a destination is permitted. The third layer is different in kind: it is a decision made by the API server about a bearer token, and no amount of manifest content can argue with it. This is the layer worth spending effort on, because it is the only one that holds when the others have been talked around.

Finding the identity that actually applies

Both major controllers reconcile as a Kubernetes ServiceAccount. Before anything else, establish which account and what it holds.

kubectl -n flux-system get serviceaccount
kubectl auth can-i --list \
  --as=system:serviceaccount:flux-system:kustomize-controller

The same question for Argo CD is asked against its application controller account in the Argo CD namespace. Two results are common and both matter:

• A long list ending in a wildcard entry means the account is bound to cluster-admin, directly or through an aggregated role.
• A short list scoped to Flux or Argo CD custom resources means the controller manages its own objects and impersonates something else to apply workloads — which is the shape you want.

flowchart TB
    A["Rendered manifest"] --> B["Controller process"]
    B -->|"applies as"| C["Reconciling ServiceAccount"]
    C --> D["API server authorisation"]
    D -->|"allowed verbs only"| E["Object written"]
    D -->|"forbidden"| F["Reconciliation fails, object unchanged"]
    G["Role and RoleBinding in namespace"] --> D
    H["ClusterRole and ClusterRoleBinding"] --> D

Scoping the identity

The scoped pattern gives each tenant namespace its own ServiceAccount, binds it to a Role in that namespace only, and tells the controller to reconcile that tenant as that account.

apiVersion: v1
kind: ServiceAccount
metadata:
  name: payments-reconciler
  namespace: payments
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: payments-reconciler
  namespace: payments
rules:
  - apiGroups: ['', 'apps', 'networking.k8s.io']
    resources: ['configmaps', 'services', 'deployments', 'ingresses']
    verbs: ['get', 'list', 'watch', 'create', 'update', 'patch', 'delete']
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: payments-reconciler
  namespace: payments
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: payments-reconciler
subjects:
  - kind: ServiceAccount
    name: payments-reconciler
    namespace: payments

kubectl apply -f rbac.yaml

Three properties of this Role are deliberate. It names resources rather than using a wildcard, so a chart that suddenly wants a ValidatingWebhookConfiguration fails instead of succeeding. It is a Role, not a ClusterRole, so nothing it grants applies outside the namespace. And it omits secrets and RBAC resources, which are the two categories that turn a namespace-scoped account into a lateral movement tool.

Impersonation moves the decision

Both controllers support reconciling as an account other than their own. In Flux, a Kustomization carries spec.serviceAccountName, and the kustomize-controller impersonates that account for every apply it performs for that Kustomization. The lockdown flag --default-service-account makes an unspecified account default to a named one in the tenant namespace rather than to the controller identity, which is the difference between a missing field being safe and a missing field being cluster-admin.

The important consequence is architectural: with impersonation, the tenant boundary is expressed in Kubernetes RBAC and enforced by the API server. Without it, the boundary is expressed in controller configuration and enforced by the controller, which means a controller bug, a mis-scoped custom resource, or a cross-namespace reference can cross it.

Auditing what you have

The audit is mechanical and should be automated:

• For each reconciling account, produce the effective verb list and diff it against the intended one.
• Flag any binding to cluster-admin, to an aggregated admin role, or to a ClusterRole with wildcard resources.
• Flag any account that can write RBAC objects or read secrets outside its own namespace.
• Confirm every tenant custom resource names an explicit service account, and that the default for a missing field is a powerless account rather than the controller.

Production discipline

1. No tenant reconciler is bound to a ClusterRole. If it needs cluster-scoped objects, that is a platform path, not a tenant path.
2. The default for a missing service account is powerless. Configure it explicitly rather than relying on the controller identity.
3. Effective permissions are asserted in CI. A test that runs the permission listing against expected output catches the widening that happens during incidents.
4. The platform path keeps its privileged identity and its own review. Bootstrap genuinely needs broad rights; the answer is to isolate it, not to pretend it does not exist.

Cross-course references

• Kubernetes for Production Sysadmins - Part LVIII (RBAC), Part LX (ServiceAccounts) and Part CVII (MultiTenancy) cover the primitives and the tenancy models this lesson composes.
• Git, CI/CD & GitOps for Infrastructure Engineers - Part XCI (LeastPrivilege) covers the same principle applied to CI identities rather than reconciling ones.
• Terraform for Production Sysadmins - Part XIX (Security) covers scoping the apply identity where the target is a cloud API.

Quiz