Skip to main content
RunBook Academy

KubernetesIII · Kubernetes APIKubernetes API

Authentication — who is making this request?

Intermediate⏱ ~16 minkubectl

What you'll learn

  • Identify the authentication methods the API server supports and when each is appropriate
  • Explain how ServiceAccount projected tokens work and why they replaced long-lived tokens
  • Configure OIDC integration for human access and evaluate its trade-offs
  • Diagnose authentication failures in production

Prerequisites

Verified against Kubernetes 1.34.x · kubeadm 1.34.x · kubectl 1.34.x · etcd 3.6.x · CoreDNS 1.11.x · containerd 1.7.x / 2.x · 2026-08-16

Not yet marked complete on this device.

The API server’s authentication layer answers one question: who is making this request? The answer becomes the UserInfo passed to authorisation and admission. This lesson covers the supported authentication methods, the trade-offs between them, and how to debug the failures each one produces.

The authenticator chain

The API server is configured with one or more authenticators. For each request, it tries every configured authenticator in order; the first that recognises the credentials returns a UserInfo. If no authenticator returns a UserInfo, the request is rejected (401) or treated as anonymous (system:anonymous).

flowchart LR
    R[Request] --> X[X.509 client cert]
    X -->|no match| B[Bearer token]
    B -->|no match| S[ServiceAccount token]
    S -->|no match| O[OIDC]
    O -->|no match| W[Webhook token auth]
    W -->|no match| AN[Anonymous]

The chain is configured by --authentication-config (a single file in 1.34+; replaces --client-ca-file, --token-auth-file, --service-account-key-file, --api-audiences, etc.). Production clusters use a structured config file with TokenReview authentication for OIDC and projected ServiceAccount tokens.

X.509 client certificates

The TLS handshake itself can authenticate the client. The API server is configured with a CA bundle (--client-ca-file or in the auth-config); the client presents a certificate signed by that CA.

# Subject of the client certificate
CN = admin-prod                # → username
O = system:masters             # → groups

A 1.34 cluster maps:

  • CN (Common Name) → UserInfo.username
  • O (Organisation) values → UserInfo.groups

This is the most common authentication method for:

  • Bootstrapping a cluster (kubeadm uses client certs for the initial admin)
  • Legacy ServiceAccounts (long-lived; deprecated in 1.24+)
  • Custom integrations (operators that authenticate via cert)

Production trade-offs:

  • Pro: simple, no central service required
  • Con: cert management is operational pain (rotation, revocation, distribution)
  • Con: a leaked cert cannot be revoked without rotating the CA
  • Con: difficult to centralise audit (who has which cert?)

Bearer tokens

A bearer token is a string sent as Authorization: Bearer <token>. The API server validates the token’s signature against a configured key set and extracts the UserInfo from the token’s claims.

Token types:

  • ServiceAccount projected tokens — JWT signed by the API server’s service account key.
  • OIDC tokens — JWT signed by the configured OIDC issuer.
  • Legacy bearer tokens — long-lived opaque tokens stored in Secrets; deprecated in 1.24+.
  • Bootstrap tokens — short-lived tokens used during node bootstrap; ephemeral.

ServiceAccount projected tokens

Modern ServiceAccount tokens are projected by the kubelet into the Pod’s filesystem:

sequenceDiagram
    autonumber
    participant Pod as Pod
    participant K as kubelet
    participant API as API server
    participant T as TokenRequest API

    K->>API: Create Pod (with serviceAccountName)
    API->>T: Issue projected token for this Pod
    T->>API: sign JWT with SA key, exp=1h
    T-->>API: token\nK->>Pod: mount /var/run/secrets/kubernetes.io/serviceaccount/token
    Note over Pod: App uses token to call API
    loop every 1h\n        K->>API: rotate (get new token, write to mount)
    end

Projected tokens:

  • Are signed JWTs with a configurable TTL (default 1 hour)
  • Include the ServiceAccount’s identity, namespace, and bound audience
  • Are mounted at a well-known path
  • Are automatically rotated by kubelet before expiry

The Pod’s mount path:

/var/run/secrets/kubernetes.io/serviceaccount/
├── ca.crt           # CA bundle to verify the API server
├── namespace        # the Pod's namespace
└── token            # the projected JWT

Application code reads token and uses it as a bearer token when calling the API server.

OIDC tokens

For human access, the API server integrates with OIDC. The cluster trusts tokens signed by the configured OIDC issuer:

# /etc/kubernetes/auth-config.yaml (1.34+)
kind: AuthenticationConfiguration
apiVersion: apiserver.config.k8s.io/v1beta1
jwt:
- issuer:
    url: https://dex.example.com
    audiences:
    - kubernetes
  userNameClaim: email
  groupsClaim: groups

The user authenticates to the IdP (Dex, Okta, Azure AD, etc.) and the IdP returns a token. kubectl passes that token as Authorization: Bearer <token> to the API server. The API server validates the signature against the IdP’s JWKS endpoint, extracts the username and groups from the claims, and proceeds.

Production trade-offs:

  • Pro: short-lived tokens (default 1 hour); MFA enforced by the IdP; central audit; easy revocation
  • Pro: no certificate management for humans
  • Con: requires a reliable IdP; IdP outage = cluster authentication outage
  • Con: IdP misconfiguration can grant excessive access

Bootstrap tokens

Used during kubeadm join. A bootstrap token is a short-lived token created by kubeadm token create and used to authenticate the joining node. The token has a TTL (default 24 hours) and is bound to a specific joining operation.

Production rarely interacts with bootstrap tokens directly; they are managed by kubeadm.

Webhook token authentication

For integrations with non-OIDC systems (LDAP, custom auth), the API server can call an external token reviewer:

kind: AuthenticationConfiguration
apiVersion: apiserver.config.k8s.io/v1beta1
webhook:
- apiVersion: authentication.k8s.io/v1beta1
  kubeConfigFile: /etc/kubernetes/webhook-auth-kubeconfig
  tokenReview:
    enabled: true

Each bearer token is sent to the webhook; the webhook returns authenticated: true or false. This is the way to integrate custom enterprise identity systems.

Anonymous authentication

If --anonymous-auth=true (default in older clusters; often disabled in production), requests without credentials are treated as system:anonymous and system:unauthenticated. RBAC can be configured to allow or deny.

Production: disable anonymous auth (anonymous: enabled: false in 1.34+ config). Any legitimate anonymous read (e.g., the /healthz endpoint) should be configured explicitly.

How to diagnose authentication failures

“401 Unauthorized” on a kubectl command

The credentials in the kubeconfig are not accepted. Diagnose:

kubectl config view --minify
kubectl auth whoami    # requires the auth-whoami subresource

Common causes:

  • Wrong kubeconfig (a stale file, a mismerged file from KUBECONFIG=a:b:c)
  • Token expired (OIDC tokens typically have 1h TTL)
  • Certificate expired (x509: certificate has expired)
  • Wrong cluster CA (x509: certificate signed by unknown authority)

“x509: certificate has expired or is not yet valid”

Either the client cert or the API server’s serving cert has expired. Check from the operator’s machine:

openssl x509 -in ~/.kube/client.crt -noout -dates

For the API server cert, check from a node that has the CA:

openssl s_client -connect api.prod.example.com:6443 \
  -showcerts < /dev/null 2>&1 | openssl x509 -noout -dates

“JWT token is expired”

Projected token past its TTL. The application must read a fresh token from the mounted path; restarting the Pod usually restores access.

“OIDC discovery failed”

The API server cannot reach the OIDC issuer. Check:

  • Network from the API server to the issuer URL
  • Issuer URL is reachable (DNS, firewall, TLS)
  • --oidc-issuer-url and --oidc-client-id are correct

Production patterns

Pattern 1: OIDC for humans, projected tokens for workloads

# API server flags
--authentication-config=/etc/kubernetes/auth-config.yaml

The auth-config has two entries:

  • JWT for OIDC (humans authenticate via the IdP)
  • Webhook or built-in for projected tokens (workloads use ServiceAccount)

This is the standard production pattern in 2026.

Pattern 2: All-OIDC, no client certs for humans

# Only OIDC; admin uses OIDC group membership to gain cluster-admin

--client-ca-file is configured for kubelet and other control-plane clients, but humans use OIDC exclusively.

Pattern 3: No OIDC, internal CA only (air-gapped)

Some production clusters cannot reach an external IdP. The authentication method is client certificates issued by the cluster’s internal CA, with rotation handled by a custom script. This is rare and operationally expensive.

Cross-course references

  • The Linux course part XXVI-Linux-SSH covers certificate- based authentication primitives that map onto X.509 authentication.
  • The Linux course part XXVII-Linux-Auth covers central identity (LDAP, Kerberos) which is often the backend for OIDC.
  • The Observability course part V-Observability-PromArchitecture covers the metrics the API server exposes about its own authentication pipeline.
  • The Docker course part XXXVIII-Docker-Secrets covers secret management patterns that interact with ServiceAccount token distribution.

Quiz

Knowledge check · 4 questions

  1. Q1. Which Kubernetes ServiceAccount token mechanism is recommended for production workloads in 2026?

  2. Q2. By default, the Kubernetes API server disables anonymous authentication.

  3. Q3. An application is making API calls from inside a Pod using a hard-coded bearer token. After 1 hour, the calls start failing with `401 Unauthorized`. Diagnose and remediate.

    Application code: ```python # (simplified) TOKEN = "eyJhbGciOiJSUzI1NiIs..." # hard-coded r = requests.get( "https://kubernetes.default.svc/api/v1/namespaces/prod/pods", headers={"Authorization": f"Bearer {TOKEN}"}, verify="/var/run/secrets/kubernetes.io/serviceaccount/ca.crt", ) ``` Symptoms: ``` 14:00:00 app.log: GET /api/v1/... 200 OK 15:00:00 app.log: GET /api/v1/... 401 Unauthorized 15:00:00 app.log: JWT token is expired ``` Pod spec: ```yaml apiVersion: v1 kind: Pod metadata: name: my-app spec: serviceAccountName: my-app containers: - name: my-app image: my-app:v1 ``` The Pod's mounted ServiceAccount token (`/var/run/secrets/kubernetes.io/serviceaccount/token`) is rotated every 1 hour by kubelet.

  4. Q4. Compare X.509 client certificate authentication and OIDC token authentication for human access. When would you choose each in production?

Passing score: 75%. Answers are checked in this browser.

Production discipline

  • Use OIDC for humans and projected ServiceAccount tokens for workloads. This is the standard production pattern.
  • Disable anonymous auth (--anonymous-auth=false). Any legitimate anonymous access should be configured explicitly per endpoint.
  • Treat the IdP as a critical dependency with HA, monitoring, and an SLO that matches the cluster’s.
  • Rotate client certs before they expire; track expiry in a calendar.
  • For workload tokens, read at request time, not at Pod start. Hard-coded or cached tokens expire within an hour.