Skip to main content
RunBook Academy

Git, CI/CD & GitOpsXCIV · Incident: Secret LeakIncidentResponse

Assess exposure — who had access and what was used in the window

Advanced⏱ ~26 mingit

What you'll learn

  • Enumerate every internal system that holds a copy of the repository in the leaked state
  • Identify the external surfaces — forks, mirrors, archives, search indexes — that may have indexed the leaked commit
  • Determine the credential-use window from the commit time to the disable time and audit every API call in that window
  • Produce an exposure report that distinguishes reachable credentials from used credentials

Prerequisites

Verified against Git 2.55.x teaching target; 2.40+ minimum · GitHub Actions continuous service; Aug 2026 documentation baseline · Argo CD v3.5.x teaching target; v3.0+ minimum · Flux v2.9.x · Sigstore Cosign v3.1.x · SLSA v1.2 · OCI Distribution Specification v1.1 · Git LFS v3.7.1 · Kubernetes (cross-course target) 1.36.x

Not yet marked complete on this device.

Once the credential is disabled and rotation is underway, the next step is exposure assessment. The question is “who had access, and what was used in the window?”.

The three exposure surfaces

A leaked credential lives on three surfaces:

flowchart LR
    A["leaked commit"] --> B["internal copies"]
    A --> C["external surfaces"]
    A --> D["credential-use window"]
    B --> B1["CI runners with checkout"]
    B --> B2["developer laptops"]
    B --> B3["backup snapshots"]
    B --> B4["internal mirrors"]
    C --> C1["public forks"]
    C --> C2["archive.org snapshots"]
    C --> C3["search engine cache"]
    C --> D1["git host search index"]
    D --> D2["API calls during window"]
  • Internal copies. CI runners, developer laptops, backup snapshots, internal mirrors. Every system that ran git clone after the commit.
  • External surfaces. Public forks, archive.org, search engine caches, the git host’s search index. Outside the team’s control.
  • Credential-use window. Every API call made by the credential between commit time and disable time.

Enumerating internal copies

For each system class:

  • CI runners. Workspace is wiped between jobs, but cache, image layers, and log archive retain the checkout.
  • Developer laptops. Every developer who cloned the repository holds a copy.
  • Backup snapshots. Every snapshot taken between commit time and disable time contains the secret.
  • Internal mirrors. Every mirror that syncs the repository holds a copy.

Enumerating external surfaces

External surfaces are enumerated by searching for the credential fingerprint on each surface. Public forks are searchable through the git host’s search index and external scanners. Archive.org snapshots may contain the file. Search engine caches retain content until the cache TTL expires. The git host’s code search may return the file even after the repository is made private.

The credential-use window

The window is bounded by two timestamps and audited by the cloud provider’s audit log:

  • Start. The commit time, or the earliest time the credential could have been extracted from a clone.
  • End. The disable time.

AWS: CloudTrail by access key ID. GCP: Cloud Audit Logs by service account email. Azure: Activity Log by service principal. GitHub: audit log by user or PAT.

Producing the exposure report

Three sections: internal copies (system class, count, action required), external surfaces (each surface, takedown status), and credential use (audit log summary, count of API calls, resources touched, any call that suggests unauthorised use).

Production discipline

  1. Enumerate before remediating. Internal copy and external surface lists must be complete before cleanup begins.
  2. The use window is the truth. Treat the audit log result as the source of truth.
  3. Document the report. A report that exists only in chat cannot be audited.

Cross-course references

  • Linux for Production Sysadmins - Part XXX (AuditAndLog) covers the cloud audit log patterns.
  • Ansible for Production Sysadmins - Part XXXVII (RepoArch) covers the backup and mirror patterns.
  • Terraform for Production Sysadmins - Part XII (State) covers state-file backup enumeration.

Quiz

Knowledge check · 4 questions

  1. Q1. A leaked credential has been disabled. The next response step is to enumerate the internal copies of the secret. Which system class is most likely to hold a copy that the team forgets to audit?

  2. Q2. The exposure class of a leak is determined by the commit time of the leaked commit.

  3. Q3. Name the three surfaces the exposure assessment enumerates.

  4. Q4. An AWS access key has been leaked in a public repository. The key has been disabled. Triage the exposure assessment and identify the surface most likely to change the response class from exposure to compromise.

    The commit was made three weeks ago. The key is held by a single Terraform automation pipeline. The key has been disabled. The audit log query must determine whether the key was used between the commit time and the disable time.

Passing score: 75%. Answers are checked in this browser.