Production credentials on runners — why long-lived credentials on shared runners are catastrophic

A credential on a shared CI runner is a credential whose lifetime is the runner lifetime, whose scope is the runner scope, and whose readers are every job the runner has ever executed. A static AWS access key in ~/.aws/credentials on a self-hosted runner is a credential that, the moment it lands, begins accumulating readers: job 1 reads it, job 17 reads it, the next thousand jobs read it. The structural fix is OIDC federation: a token issued per job, scoped to the role the job needs, and dead when the job ends.

The blast radius of a long-lived credential

A long-lived credential is a string with no expiry and no scope. On a CI runner, the consequences compound:

flowchart LR
    K["Long-lived key\nAKIA... in ~/.aws/credentials"] --> R["Runner host"]
    R --> J1["Job 1\nreads the key"]
    R --> J2["Job 17\nreads the key"]
    R --> J3["Job 482\nreads the key"]
    R --> JN["Job N\nattacker exfiltrates"]
    JN --> E["All credentials\nevery system the role can reach"]

• Lifetime. Valid for as long as the team remembers to rotate it. A typical cadence is quarterly or annual; a compromised credential is valid for that entire window.
• Scope. Whatever IAM permissions the key was granted. An account-wide AdministratorAccess key on a runner is an AdministratorAccess key that every job inherits.
• Readers. Every job the runner has ever executed can read the credential. Persistent hosts carry the credential across job boundaries; the attacker’s job reads and exfiltrates.

The credential’s blast radius is the union of every reader and every IAM permission the key holds. For a typical team, that union is “every system the team owns”.

Static keys versus OIDC federation

flowchart TB
    subgraph STATIC["Static key model"]
        S1["Long-lived AWS access key\nstored on runner"] --> S2["Every job reads it"]
        S2 --> S3["Compromise = key valid until rotation"]
    end
    subgraph OIDC["OIDC federation model"]
        O1["Forge issues ID token\nper job, signed"] --> O2["Cloud validates token, role, repo, branch"]
        O2 --> O3["Cloud issues short-lived STS token\n(~1 hour)"]
        O3 --> O4["Job uses STS token;\ntoken dies with job"]
    end

• Static key. The team creates an IAM user, generates an access key, stores the key on the runner (in ~/.aws/credentials, in a CI secret), and uses the key for every job. The key has no expiry; the team rotates it on a schedule.
• OIDC federation. The forge (GitHub Actions, GitLab CI) issues a signed ID token per job. The token is bound to the workflow run, the repository, the branch, and the workflow file. The cloud provider’s trust policy validates the token and issues a short-lived STS token (typically valid for 1 hour).

The static key is a property of the runner; OIDC is a property of the job. A runner with OIDC has no credentials; a job on that runner has credentials for the duration of the run.

The lifetime principle

The principle that ties the two models together: the credential’s lifetime must be no longer than the workload’s lifetime.

flowchart LR
    JOB["Job lifetime\n(minutes)"] --> CRED["Credential lifetime\n(must be ≤ job lifetime)"]

Three operational consequences follow:

• No long-lived keys on the runner. The runner has no IAM user, no static access keys, no service account JSON files. Credentials arrive per job and leave when the job ends.
• Per-job token issuance. Each job that needs cloud access requests a token from the cloud trust policy. The token is scoped to the role the workflow declared.
• No shared credentials between jobs. Job 1 cannot pass a credential to job 2 because the credential is dead at the end of job 1. The cross-job exfiltration window is closed by construction.

Practical OIDC configuration

A GitHub Actions workflow that uses OIDC for AWS:

# Workflow file
permissions:
  id-token: write   # Required to request the OIDC token
  contents: read

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Configure AWS credentials via OIDC
        uses: aws-actions/configure-aws-credentials@v4
        with:
          role-to-assume: arn:aws:iam::"$AWS_ACCOUNT_ID":role/github-actions-deploy
          aws-region: us-east-1
      - name: Deploy
        run: aws s3 sync ./build s3://"$BUCKET_NAME"

The cloud-side trust policy gates the role to a specific repository and branch:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Federated": "arn:aws:iam::123456789012:oidc-provider/token.actions.githubusercontent.com"
      },
      "Action": "sts:AssumeRoleWithWebIdentity",
      "Condition": {
        "StringEquals": {
          "token.actions.githubusercontent.com:aud": "sts.amazonaws.com",
          "token.actions.githubusercontent.com:sub": "repo:acme/infra:ref:refs/heads/main"
        }
      }
    }
  ]
}

A workflow on a different repo or branch cannot assume the role. The STS token is valid for one hour; the job uses it; the job ends; the token dies.

When OIDC is not available

Some cloud providers do not have a forge-side OIDC integration, or the team’s account tier does not support it. The fallback is per-deploy scoped credentials with strict rotation:

• Short-lived service account tokens. GCP service account keys with 10-minute expiry, Azure managed identities with 24-hour rotation.
• Vault-issued dynamic secrets. HashiCorp Vault issues short-lived database credentials, cloud credentials, and SSH keys per CI job. The credential’s lifetime is the lease; the lease expires.
• Per-deploy manual rotation. A team that cannot use OIDC or dynamic secrets must rotate the credential after every deploy. The rotation cadence becomes the credential’s lifetime.

The fallback is not equivalent to OIDC; it is a smaller window on the same problem.

Production discipline

1. No long-lived cloud credentials on runners. No IAM user keys, no service account JSON files, no static credentials on disk or in environment variables.
2. OIDC federation for cloud access. Each cloud that the CI uses has an OIDC integration; each workflow declares the role it needs; each role’s trust policy is scoped to repo, branch, and workflow path.
3. Per-job token issuance; per-job revocation. The token’s lifetime is the job’s lifetime; the token dies when the job ends. No token reuse across jobs.
4. Audit the credential surface quarterly. Verify each credential the runners have access to is OIDC or dynamic; rotate any static credential found.

Cross-course references

• Git, CI/CD & GitOps — Part XXXIV-04 (Credential storage and rotation) covers the rotation cadence that bounds the static-key window.
• Git, CI/CD & GitOps — Part XXXVIII-05 (Environment variables and config) covers the secrets plane the runner reads from.
• AWS for Production Sysadmins — Part XXXI (IAM) covers the trust policies that gate the OIDC role assumption.
• Terraform for Production Sysadmins — Part XXII (StateSecrets) covers the analogous pattern for Terraform state credentials.

Quiz