Secret rotation cadence — when to rotate, what triggers rotation, and the discipline

A long-lived credential has a rotation cadence, and the cadence is the credential’s blast radius ceiling. The attacker who exfiltrates a quarterly-rotated key has 90 days of uninterrupted access; the attacker who exfiltrates an annual-rotated key has a year. The cadence is the window, and the window is the consequence. The discipline is to set the cadence below the consequence the team is willing to absorb, and to have a runbook for the out-of-cycle trigger that forces rotation outside the cadence.

The cadence is a blast radius ceiling

The relationship between cadence and blast radius is direct. A credential that rotates quarterly has a maximum exposure window of 90 days per leak. A credential that rotates annually has a maximum window of 365 days. A credential that never rotates has an unbounded window.

flowchart LR
    CRED["Long-lived credential"] --> CAD{"Rotation\ncadence?"}
    CAD -->|"never"| UB["Unbounded window"]
    CAD -->|"annual"| ANN["365-day window"]
    CAD -->|"quarterly"| Q["90-day window"]
    CAD -->|"monthly"| M["30-day window"]
    CAD -->|"per deploy"| D["Deploy window\n(hours)"]

The right cadence is the cadence whose window is below the consequence the team is willing to absorb. A production IAM access key whose blast radius is the entire production account needs a cadence whose window is shorter than the team’s mean-time-to-detect a leak. If the team detects a leak in 24 hours, a 30-day cadence is too long.

The triggers for out-of-cycle rotation

The cadence is the steady-state rotation. The triggers are the events that force rotation outside the cadence. Every team needs a list of triggers; every list has the same first three entries:

• Leak. A credential that has appeared in a log, an artifact, a commit, a chat message, a screenshot, or any other sink whose reader set the team cannot enumerate is a credential that must be rotated now. The rotation is at the source (the IAM role, the vault, the registry); the CI secret is overwritten with the new value.
• Role change. An employee who changes roles - who leaves the team, who moves to a different project, who is promoted to a less-technical role - is an employee whose credentials should be rotated. The credential may not have leaked; the credential’s reader set has changed.
• Employee offboarding. An employee who leaves the organisation is an employee whose every credential must be rotated. The offboarding checklist is the rotation trigger; the offboarding runbook is the rotation runbook.
• Dependency compromise. A third-party dependency - a library, a tool, a CI action - that has been compromised is a dependency whose credentials should be rotated. The dependency may have read the credential during the compromise window.
• Scope change. A credential whose permissions have changed - whose role was widened, whose IAM policy was modified, whose vault token was granted a new policy
  • is a credential whose lifetime should be reset. The rotation aligns the credential with its new scope.

The trigger list is operational, not theoretical. Every entry corresponds to an event the team has either experienced or should expect to experience.

Rotation versus revocation

The two operations look similar and behave differently. Revocation is “this credential no longer grants access”; rotation is “this credential’s value is replaced with a new one, and the old value no longer grants access”.

flowchart LR
    A["Leak detected"] --> B{"Response?"}
    B -->|"Revoke"| R["Disable old value\nat the source"]
    B -->|"Rotate"| ROT["Issue new value\nat the source"]
    ROT --> R2["Disable old value\nat the source"]
    ROT --> C["Overwrite CI secret\nwith new value"]
    R --> C2["CI secret still holds\nold (now invalid) value"]
    R2 --> C
    C --> V["Verify new value works"]
    V --> D["Audit sinks for\nresidual leak"]

Rotation is revocation plus replacement. A team that only revokes leaves the CI store with an invalid value the runner will fail on; a team that only rotates leaves the old value valid at the source. The correct response to a leak is rotation: issue new, disable old, overwrite CI secret, verify, audit.

The rotation runbook

A predictable rotation is a runbook the team can follow without thinking. The runbook has six steps; every step has an owner; every step produces an artifact the auditor can read.

1. Identify the credential. The runbook names the source system (IAM role, vault path, registry account) and the CI secret name. The team knows exactly which credential is rotating.
2. Issue the new value at the source. The new value is generated by the source system, not by a human. A human-generated value is a value with lower entropy and a value whose generation is not auditable.
3. Disable the old value at the source. The old value is revoked before the CI secret is overwritten. The window between new-value-issue and old-value- revoke is a window both values are valid; the runbook minimises the window.
4. Overwrite the CI secret. The CI store receives the new value; the runner reads the new value on the next job.
5. Verify the new value works. A smoke-test workflow reads the new value and exercises the credential’s primary use. The verification step catches a typo in the overwrite, a source system that did not issue the value correctly, or a CI store that did not propagate the new value.
6. Audit the sinks. The runbook names the sinks the credential may have reached (logs, artifacts, error trackers) and the procedure for scrubbing them. The audit is the closure of the leak window.

The runbook is a checklist, not a guideline. A checklist the team can execute under pressure is a checklist that closes the rotation window; a guideline the team has to interpret is a guideline that delays the rotation.

Cadence by credential type

Different credentials need different cadences. The rule is: the broader the credential’s blast radius, the shorter the cadence.

flowchart LR
    A["Credential type"] --> B["Blast radius"]
    B --> C["Cadence"]
    A1["Production IAM key"] --> B1["Production account"]
    A1 --> C1["30-90 days\nor OIDC"]
    A2["Staging IAM key"] --> B2["Staging account"]
    A2 --> C2["Quarterly"]
    A3["Container registry token"] --> B3["Image push/pull"]
    A3 --> C3["Quarterly"]
    A4["Notification webhook"] --> B4["Chat room"]
    A4 --> C4["Annually or on\nemployee offboarding"]
    A5["Signing key"] --> B5["Artifact integrity"]
    A5 --> C5["Per release"]

The cadence is a function of the blast radius and the mean-time-to-detect. A team whose detection window is 24 hours needs a rotation window shorter than 24 hours to close the gap; OIDC closes the gap by removing the rotation window from the structural design. Lesson XLII-06 returns to OIDC.

Production discipline

1. Cadence by blast radius, not by calendar. The production credential rotates more often than the staging credential; the staging credential rotates more often than the notification webhook.
2. Out-of-cycle triggers are documented and exercised. Leak, role change, offboarding, dependency compromise, scope change. The runbook covers each.
3. Rotation is at the source. The CI store is overwritten after the source system issues the new value; the old value is revoked before the CI secret is updated.
4. Verification is a step, not an assumption. A smoke-test workflow reads the new value and exercises it. The verification catches the typo, the wrong value, the unpropagated update.
5. Sink audit is part of the runbook. The rotation closes the credential’s validity window; the audit closes the disclosure window. Both must close.

Cross-course references

• Git, CI/CD & GitOps — Part XXXV-03 (The rotation response) covers the rotation procedure in detail.
• Git, CI/CD & GitOps — Part XXXIV-04 (Credential storage and rotation) covers the credential-store side of the same discipline.
• Git, CI/CD & GitOps — Part XLI-03 (Production credentials on runners) covers why OIDC is the structural upgrade.

Quiz