Kubernetes access from CI — what cluster credentials do

A Kubernetes credential on a CI runner is a production credential: the namespace the token can read is the namespace the cluster exposes, the workload it can apply is the workload the cluster runs, and the secret it can mount is the secret the pod consumes. A CI job that holds a cluster-admin kubeconfig holds every secret in every namespace. The discipline is to scope the credential narrowly — namespace-scoped service-account tokens, not cluster-wide kubeconfigs — and to verify the scope with kubectl auth can-i before every privileged action.

What a cluster credential enables

A Kubernetes credential is a token plus a kubeconfig. The token authorises API calls; the kubeconfig names the cluster, the namespace, and the user. From a CI runner, the credential enables four categories of action:

flowchart TB
    K["Cluster credential"]
    K --> R1["Read"]
    K --> R2["Write"]
    K --> R3["Exec"]
    K --> R4["Secrets"]
    R1 --> N["Namespace contents"]
    R2 --> M["Mutations"]
    R3 --> D["Debug"]
    R4 --> S["Sensitive data"]

The categories compound. A credential that can read secrets can read the database URL; a credential that can exec can read the pod’s environment; a credential that can write can replace the deployment with a hostile image. The blast radius is the union of every action the token can perform.

Verify cluster access before every action

The first step after acquiring a cluster credential is to verify what the credential can do. Kubernetes provides kubectl auth can-i:

kubectl auth can-i create deployments.apps --namespace=production

The output is yes or no. The CI engineer writes a test that runs the verification before every privileged action and fails the workflow on no. The verification catches a wrong kubeconfig, a token scoped to the wrong namespace, a RoleBinding that was deleted, and a leaked credential that has been revoked.

CI-direct versus GitOps-mediated

There are two patterns for cluster access from CI, with different blast radii:

flowchart TB
    subgraph DIRECT["CI-direct"]
        CI1["CI job"] -->|"kubectl apply"| K1["Cluster API"]
    end
    subgraph GITOPS["GitOps-mediated"]
        CI2["CI job"] -->|"push manifest"| R["Git repository"]
        R -->|"reconciles"| K2["Cluster API"]
    end

• CI-direct. The CI runs kubectl apply. The cluster credential is on the runner; the blast radius is whatever the credential can reach.
• GitOps-mediated. The CI commits a manifest; the GitOps controller (Argo CD, Flux) reconciles the cluster. The CI has no cluster credential; the blast radius is whatever a malicious commit can cause, bounded by the GitOps controller’s RBAC.

The GitOps-mediated pattern is structurally smaller. The CI cannot apply directly; the controller is the only writer. CI-direct access is necessary for one-off operations (debug exec, log streaming, emergency rollback) but should be the exception, not the rule.

Short-lived tokens for CI-direct access

When CI-direct access is necessary (debug, exec, log collection), the structural fix is short-lived tokens:

kubectl create token ci-debug --namespace=production \
  --duration=10m --audience=api

The token’s lifetime is the operation’s lifetime. The token expires when the operation completes; the runner does not carry a credential between jobs. The BoundServiceAccountToken projection in Kubernetes 1.21+ provides the same property for in-cluster workloads.

Production discipline

1. No cluster-admin tokens on CI runners. The token has the cluster’s blast radius; the runner is the wrong place for that.
2. Namespace-scoped service accounts per workflow. Each workflow gets its own service account, its own RoleBinding, its own namespace.
3. kubectl auth can-i before every privileged action. Verification catches the wrong scope, the wrong namespace, the revoked token.
4. GitOps mediation as the default. CI writes to Git; the controller reconciles. CI-direct is the exception.
5. Short-lived tokens for CI-direct access. Token lifetime is the operation’s lifetime.

Cross-course references

• Part XC-01 (What the CI platform can touch) maps the cluster surface in the wider blast radius.
• Part LII-01 (The Kubernetes CI discipline) covers the manifest-validation pattern GitOps mediation relies on.
• Part XXXVII (GitOps patterns) covers Argo CD and Flux reconciliation.
• Kubernetes for Production Sysadmins — Part LVI (DefenseInDepth) covers cluster-side defences.

Quiz