Token volume projection — the SA token in the Pod

The projected token volume is how a Pod accesses its ServiceAccount’s credentials. The kubelet requests a token via TokenRequest, mounts it at a well-known path, and rotates it before expiry. The workload reads the file, presents the token, and authenticates. This lesson covers the volume’s contents, the rotation lifecycle, and the production patterns for explicit projection.

The default mount

When a Pod uses a SA, the SA admission controller adds the projected token volume to the Pod spec:

spec:
  volumes:
  - name: kube-api-access
    projected:
      sources:
      - serviceAccountToken:
          path: token
          expirationSeconds: 3600
      - configMap:
          name: kube-root-ca.crt
          items:
          - key: ca.crt
            path: ca.crt
      - downwardAPI:
          items:
          - path: namespace
            fieldRef:
              fieldPath: metadata.namespace
  containers:
  - name: api
    volumeMounts:
    - name: kube-api-access
      mountPath: /var/run/secrets/kubernetes.io/serviceaccount
      readOnly: true

The volume has three sources:

1. ServiceAccountToken — the projected JWT. Path is token; default expiry is 3600 seconds (1 hour).
2. ConfigMap kube-root-ca.crt — the cluster’s CA bundle. Path is ca.crt.
3. DownwardAPI — the Pod’s namespace. Path is namespace.

flowchart LR
    A[Kubelet] --> B[TokenRequest]
    B --> C[API server signs JWT]
    C --> D[Mount at /var/run/secrets/...]
    D --> E[Workload reads]
    E -->|token expires soon| F[Kubelet rotates]
    F --> B

Reading the volume from inside the Pod

# Inspect the directory
ls -la /var/run/secrets/kubernetes.io/serviceaccount/

# Read the namespace
cat /var/run/secrets/kubernetes.io/serviceaccount/namespace

# Read the CA
cat /var/run/secrets/kubernetes.io/serviceaccount/ca.crt

# Read the token (first 60 chars)
head -c 60 /var/run/secrets/kubernetes.io/serviceaccount/token
# eyJhbGciOiJSUzI1NiIs...

# Decode the token claims
TOKEN=$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)
echo "$TOKEN" | cut -d. -f2 | base64 -d 2>/dev/null | jq .

{
  "iss": "https://kubernetes.default.svc",
  "sub": "system:serviceaccount:prod:api-sa",
  "aud": ["https://kubernetes.default.svc"],
  "exp": 1755356400,
  "kubernetes.io": {
    "namespace": "prod",
    "serviceaccount": {
      "name": "api-sa",
      "uid": "..."
    }
  }
}

Rotation

The kubelet rotates the mounted token before it expires. The default behaviour: when the token has reached 80% of its lifetime (48 minutes for a 1-hour token), the kubelet requests a new token and atomically replaces the file.

gantt
    title Token rotation timeline
    dateFormat HH:mm
    axisFormat %H:%M
    section Lifecycle
    Token issued       :a1, 10:00, 12m
    Rotation threshold :milestone, 10:48, 0m
    Token 2 issued     :a2, after a1, 60m
    Original expires   :milestone, 11:00, 0m
    Token 3 issued     :a3, after a2, 60m

A workload that re-reads the file before each API call sees the new token transparently. A workload that caches the token value sees the rotation only when it re-reads the file.

Explicit projection

A workload can override the default mount with an explicit projected volume:

spec:
  serviceAccountName: api-sa
  volumes:
  - name: custom-token
    projected:
      sources:
      - serviceAccountToken:
          path: api-token
          audience: vault://prod.example.com
          expirationSeconds: 600
  containers:
  - name: api
    volumeMounts:
    - name: custom-token
      mountPath: /etc/secrets/vault
      readOnly: true

This configuration:

• Mounts a custom token at /etc/secrets/vault/api-token with a 10-minute expiry and an audience of vault://prod.example.com.
• Does not mount the default /var/run/secrets/kubernetes.io/serviceaccount/ unless automountServiceAccountToken: true is set on the SA or Pod.

The custom audience allows the workload to authenticate to Vault with a token that the API server would reject.

Common failure modes

1. Token file missing. The Pod’s automountServiceAccountToken is false. The workload reports “no such file.” The fix is to set it to true on the Pod or SA.
2. Token expired. The workload cached the token value and the kubelet rotated it. The workload’s API call fails with 401. The fix is to re-read the file before each call.
3. Wrong CA. The ca.crt in the volume does not match the API server’s CA. TLS verification fails. The fix is to verify the CA chain and re-mount.
4. Wrong audience. A workload uses the default-mounted token to authenticate to Vault; Vault rejects because the audience is the API server. The fix is an explicit projection with --audience.

Production failure modes

1. Workload caches the token across rotations. The workload reads the file once and caches the value in memory. At the first rotation, the cache is stale. The fix is a client library that re-reads before each call.
2. Multiple tokens mounted. A Pod with both the default mount and an explicit projection has two tokens. The workload uses the wrong one. The fix is to disable the default mount with automountServiceAccountToken: false on the Pod and use only the explicit projection.
3. Long expiry with no rotation check. A workload with a 24-hour token (custom projection) does not re-read until the expiry. If the API server’s signing key is rotated mid-window, the token is rejected. The fix is shorter expiries (10 minutes for short-lived workloads).

Cross-course references

• The Linux course covers JWT and JWKS — the primitives that the kubelet uses.
• The Observability course covers the audit log entries for projected token requests.

Quiz

Production discipline

A defensible token volume projection uses the default mount for in-cluster API access and adds explicit projections for non-API consumers (Vault, cloud providers). Workloads that cache the token value break at rotation; the fix is a client library that re-reads the file. The kubelet’s automatic rotation at 80% of the lifetime is invisible to a workload that re-reads correctly and fatal to one that does not. The discipline is to use the default mount for API access, to use explicit projections for other audiences, and to verify that the workload handles rotation.