← All runbooks in Secrets, PKI & Certificates

high risksecurity relevant~90 min

Runbook: Respond to a Credential Committed to a Git Repository

1 · Prerequisites

Confirm every item is in place before any state change.

Authority to revoke or rotate the credential yourself, or a named person on call who holds that authority and can act within minutes
Administrative access to the hosting platform for this repository, including the ability to change visibility, restrict pushes and open a support case
A working git client and git-filter-repo 2.47 or later, which is the version the GitHub sensitive-data documentation names
Read access to the audit trail of whatever the credential authenticates to, covering the whole period since the commit was pushed
A distribution channel for the replacement credential that is not the repository you are about to clean
An incident record opened before the first action, because the timeline is the durable output of this procedure

2 · Pre-checks

Read-only diagnostic commands. If any of these don't match expected output, stop and investigate further.

· Confirm the finding is a live credential, not a fixture. Scanner hits on test data and documentation samples are common. Read the file as it stood at the commit with git show COMMITSHA:path/to/file and decide whether the value would authenticate anywhere today. Anything you cannot rule out is live.
· Record the exact bytes you are hunting, once. Every later search matches literally. If the value was line-wrapped, base64 encoded or URL encoded on its way into the file, record each form as a separate string in the incident record.
· Establish when the commit was pushed, not when it was authored. git log --format='%H %aI %cI %s' -1 COMMITSHA prints both. The push is what made the value readable by others, and it can be days after the author date.
· Establish who could read the repository during that window. A public repository means the window is the whole internet. A private one still means every member, every fork, every deploy key, every CI runner that checked it out and every laptop that cloned it.
· Check whether the platform already raised an alert. Secret scanning may hold a first-seen timestamp earlier than yours. Repository push protection is disabled by default, so its silence is not evidence that nothing was pushed.
· Confirm you can reach the revocation interface for this credential right now. Finding out that the only account with that permission belongs to someone who is asleep is a discovery to make before you announce containment, not after.
· Take a mirror clone before touching anything. git clone --mirror into a directory outside every working tree. That clone is your evidence and it is the only rollback this procedure has.

3 · Procedure

Execute each step in order. Verify the expected output of a step before moving to the next.

1Revoke or rotate the credential before any other action. The GitHub guidance is unambiguous: you need to revoke and/or rotate the secret, and once it is revoked or rotated it can no longer be used for access. Rewriting history removes a value from future readers; it removes nobody's access. Do this even when the repository is private and you believe nothing fetched it.
2Prove the old value is dead from a host that took no part in the rotation. Attempt one authenticated call with it and require a refusal. A rotation you have not watched fail is a rotation you have not made.
3Publish the replacement through the secret manager the consumer already reads. Never paste it into a commit, a pull request comment, a ticket or a chat channel. The replacement travels the path the leaked one should have travelled.
4Audit use of the old credential across the full exposure window. Bound the query by the push timestamp and the confirmed revocation timestamp. You are looking for calls you cannot attribute to your own systems, which is a different question from calls that look malicious.
5Classify on the audit evidence, not on instinct. Any unattributed use turns this from a cleanup into an intrusion with its own responder. This procedure keeps running in parallel either way.
6Only now begin the cleanup, and say out loud what it is for. Everything from here removes the value from future readers of the repository. None of it reaches a copy somebody already holds, and none of it restores confidentiality.
7**Rewrite history in the mirror clone using git filter-repo --sensitive-data-removal.** Combine it with --invert-paths --path path/to/file to drop a whole file, or with the text-replacement form for a value embedded in a file you want to keep. The flag makes filter-repo fetch all refs, report the first changed commits, report LFS objects orphaned by the rewrite, and print cleanup instructions for the other copies. Do not reach for git filter-branch; the Git manual states plainly that its use is not recommended.
8**Push the rewrite with --mirror or --prune.** filter-repo warns that refs deleted by a rewrite continue to exist on the server unless you push one of those two ways. A rewrite that leaves the old refs on the server has changed nothing that matters.
9Expect the platform to refuse some namespaces, and record which. filter-repo warns that the server may not allow a force push over some refs, and review namespaces are exactly that case. Pull request references are one of the three places the GitHub documentation names as still reachable after a rewrite.
10Tell every holder of a clone to re-clone, never to pull. filter-repo is explicit that someone running git pull && git push merges the unrewritten history back and pushes everything you removed straight back in. Tags are worse: an existing clone will not take updated tags it already holds, so holders must delete their local tags first.
11Open a platform support case for the copies you cannot reach yourself. The GitHub documentation lists clones and forks, cached views addressed directly by SHA-1, and pull requests that reference the commits. Support may decline where rotation already mitigates the risk, and that refusal is a useful thing to have on record.
12Close the path that allowed the commit. Enable repository secret scanning and push protection, which is disabled by default, and add a scanner to both the pre-commit hook and the pipeline. User-level push protection is enabled by default only for public repositories, so it never covered a private one.
13Remove the reason a long-lived value sat in a file at all. Replace it with a reference the process resolves at start-up, or with federated short-lived credentials, so the next accident has nothing durable to leak.
14Write the timeline before you close. Push time, detection time, revocation time, audit window, rewrite time, notification time and the residual risk you accepted. That record is what the next reviewer reads.

4 · Verification

Confirm the procedure actually fixed the problem.

✓The provider's own interface reports the old credential as revoked or absent, read after the change rather than inferred from the command that made it.
✓An authentication attempt with the old value from a host outside the rotation path is refused, and the refusal is recorded with its timestamp.
✓The consumer that used the credential is healthy on the replacement, observed from its own health endpoint or its own log stream rather than from the deployment tool that pushed the change.
✓A fresh mirror clone taken from the server, not the one you rewrote, yields no match for any recorded form of the value under git log --all --full-history -S.
✓The refs the rewrite deleted are absent from a fresh git for-each-ref against the server, which is what proves the push used --mirror or --prune.
✓The audit trail shows no use of the old credential after the recorded revocation timestamp.
✓Secret scanning and push protection are reported enabled in the repository settings by someone other than the person who enabled them.
✓The incident record carries push time, revocation time and audit window, and the three reconcile without gaps.

5 · Rollback

If verification fails, undo the procedure in reverse order.

↶Revocation is not reversible and must not be reversed. If the replacement breaks a consumer, repair the consumer or issue a second replacement; never reinstate the leaked value to buy time.
↶If the rewritten history is wrong, restore from the mirror clone taken during the pre-checks. That mirror is the only complete copy of the pre-rewrite state under your control.
↶If the push landed partially and the server now holds a mixture of rewritten and original refs, stop all writes, restore the mirror, and repeat the rewrite in a single pass rather than patching forward.
↶If a contributor has already merged the old history back with git pull && git push, take a fresh mirror, repeat the rewrite against it, and lock pushes until every holder confirms a clean re-clone.
↶If a consumer cannot move to the replacement inside the incident window, take that consumer offline and record the decision. An offline service is recoverable; a live leaked credential is not.
↶If the platform removes cached views and that breaks an internal link or an automation that resolved a commit by SHA, repoint the automation. Do not ask for the cached view to be restored.

6 · Escalation

When the runbook isn't enough, contact:

· Any use of the credential you cannot attribute to your own systems: this is an intrusion rather than a leak. Hand it to the security incident responder and keep this procedure running alongside.
· The credential could sign, publish, deploy or issue on behalf of others, for example a deployment identity, a package publishing token or a certificate authority credential: the blast radius is every artefact it touched, and the supply chain owner belongs in the response.
· The repository was public at any point in the window: assume harvesting by automated scrapers and treat confidentiality as already lost. Notify the security owner and whoever owns data protection reporting.
· You lack authority to revoke and the person who holds it is unreachable: escalate to the on-call manager immediately. This procedure does not advance past revocation.
· The platform declines to remove cached views or review references: record the refusal, raise the residual risk to its owner, and treat the value as permanently public from that point.
· An outside party reported the leak before your own tooling did: detection failed as well as prevention, and the detection owner needs that finding with a date attached.

A credential that reached a Git repository is disclosed, not at risk of disclosure. Git is a replication system, so the question is never whether a copy exists somewhere you cannot see; it is how many copies exist and who holds them. Clones, forks, continuous integration workspaces, editor caches, backup jobs and the platform’s own object store all took a copy the moment the push completed.

That is why the ordering in this runbook is not negotiable. Revocation is the only action that removes access. Rewriting history removes the value from the next person who reads the repository, which is a different and much smaller benefit. Teams that rewrite first spend the attacker’s window arguing about git flags while the credential still works.

Hold on to one distinction for the rest of this page: containment means the credential no longer authenticates anywhere, and cleanup means the repository no longer carries the value. Containment is finished when the provider refuses the old value. Cleanup is never completely finished, and this runbook says exactly where it stops.

When this runbook applies, and when it does not

It applies when a credential, token, private key, keystore password or connection string with an embedded password has been committed and pushed to a Git repository, whether it is still on the tip of a branch or buried in history.

It does not apply when:

The value never left the machine. A commit that exists only in a local clone with no push is a working-tree problem. Amend or reset, then confirm with git log --all that nothing reachable holds it. Running the rewrite and support steps here would burn hours of an incident channel for nothing.
The value is a public key, a certificate or a fingerprint. Those are meant to be published. Revoking a live production credential because someone recognised a PEM header is a self-inflicted outage.
The credential is a certificate authority key. That has a far larger blast radius and its own procedure. Revoking one credential will not contain a key that can mint new identities.
The value was committed by an attacker rather than by a colleague. A credential appearing in a repository you did not put it in is an intrusion indicator. Preserve the repository state and escalate before you rewrite anything.

Blast radius

Action	Reversible?	What it costs if wrong
Revoking the credential	No	Every consumer still using it stops working immediately. That cost is the point of the action, not a side effect.
Taking the mirror clone	Yes	Nothing. Skipping it costs you the only rollback the procedure has.
Rewriting history	Only if you kept a copy	Every commit from the first changed one onward gets a new identifier. Open pull requests, build pins, deployment records, release notes and bisect scripts that name an old SHA all stop resolving.
Pushing with `--mirror`	No	Refs the rewrite deleted are removed from the server. If the rewrite was wrong, the server now holds the wrong history and your mirror is the only copy of the right one.
Asking the platform to purge cached views	No	Those commits stop resolving by SHA for everyone, including your own audit tooling and any link in a past incident report.

Step 1 - Establish the value, the commit and the window

Read-only / Safeon the responder workstation, against a scratch clone— Find the commits that introduced the value and both of their timestamps.

$ INCIDENT=/run/incident
install -d -m 0700 "$INCIDENT"
cd "$INCIDENT/scratch-clone"

# The commit that added the file, with author and committer times.
git log --all --full-history --diff-filter=A \
  --format='%H %aI %cI %s' -- deploy/service.env

# Every commit that changed how many times the value appears. Read the
# value from a file so it never enters shell history or the process list.
git log --all --full-history -S "$(cat "$INCIDENT/leaked-value")" \
  --format='%H %cI %s' | tee "$INCIDENT/touching-commits"

The pickaxe search with -S finds commits where the number of occurrences of the string changed, which is what identifies both the commit that added it and any commit that removed it. The committer timestamp is the one to trust for the window, and even that is only a lower bound: the push can be later still, and the push is what made the value readable by anyone else.

Step 2 - Revoke, then prove the revocation

Service impact possiblefrom a host that plays no part in the rotation— Record the revocation moment, then require the old value to be refused.

$ INCIDENT=/run/incident

# Revoke at the provider that issued the credential. There is no
# portable command for this: use that provider's own interface, and
# prefer disable-then-delete wherever it offers both.

# Record the moment revocation completed. Every audit query below is
# bounded by this timestamp.
date -u +%FT%TZ | tee "$INCIDENT/revoked-at"

# Probe with the old value. It stays in a curl config file so it never
# appears in the process list on this host.
curl --fail --silent --show-error \
   --config "$INCIDENT/old-credential.curlrc" \
   https://api.example.com/v1/whoami
echo "probe exit status: $?"

A non-zero exit status with an authentication failure is the result you want. A success means the revocation did not take effect, which happens most often because a provider created a new credential without disabling the old one. Run the probe from a machine that did not perform the rotation, so that a cached session or an ambient identity cannot make a dead credential look alive.

Step 3 - Audit the exposure window

Read-only / Safeagainst an export of the provider audit trail— Reduce the audit trail to the exposure window and list the callers.

$ INCIDENT=/run/incident
FROM=$(cat "$INCIDENT/pushed-at")
TO=$(cat "$INCIDENT/revoked-at")

# Field names differ per provider; adjust them to the exported schema.
jq --arg from "$FROM" --arg to "$TO" \
 'select(.time >= $from and .time <= $to)' \
 "$INCIDENT/audit-export.jsonl" > "$INCIDENT/window.jsonl"

# The callers seen in the window, most frequent first.
jq -r '.source_address' "$INCIDENT/window.jsonl" | sort | uniq -c | sort -rn

Compare that list against your own egress ranges and your own runner pools. The finding you are hunting is an address you cannot account for, not an address that looks suspicious. Attackers using a stolen credential usually make ordinary-looking calls, and the anomaly is the origin rather than the operation.

Step 4 - Understand what a rewrite cannot reach

Read-only / Safeagainst the mirror clone taken in the pre-checks— List every ref the server holds, including the review namespaces.

$ MIRROR=/run/incident/repo-mirror.git

# A mirror clone uses a refspec of +refs/*:refs/*, so it brings down
# whatever the server permits you to fetch, not only branches and tags.
git -C "$MIRROR" for-each-ref --format='%(refname) %(objecttype) %(objectname)' \
| tee /run/incident/all-refs

# The refs a branch-and-tag rewrite leaves behind are the interesting ones.
grep -v -e ' refs/heads/' -e ' refs/tags/' /run/incident/all-refs \
| awk '{print $1}' | sort -u

The GitHub documentation names three places a rewrite does not reach: clones and forks of the repository, cached views addressed directly by SHA-1, and pull requests that reference the commits. The listing above makes the third one concrete. Everything outside refs/heads and refs/tags is a ref you may not be permitted to force-push over, and the review namespace is the usual example.

Step 5 - Rewrite history in the mirror

Destructivein the mirror clone only, never in a working tree— Rewrite with the sensitive-data-removal mode so the cleanup instructions are produced.

$ MIRROR=/run/incident/repo-mirror.git
cd "$MIRROR"

# Confirm the tool version before trusting the mode.
git filter-repo --version

# Drop an entire file from every commit that ever carried it.
git filter-repo --sensitive-data-removal \
              --invert-paths --path deploy/service.env

# Report what changed, so the notification names real commits.
git for-each-ref --format='%(refname) %(objectname)' | head -20

The --sensitive-data-removal mode does more than the ordinary rewrite: it fetches all refs so that references outside branches and tags are included, it tracks and reports the first changed commits, it reports any large-file objects orphaned by the rewrite, and it prints instructions for cleaning up the repository you cloned from and the other clones. Use the text-replacement form instead of --invert-paths when the value sits inside a file the project still needs.

Step 6 - Publish the rewrite and chase what it missed

Destructiveagainst the origin, with pushes restricted— Push so that deleted refs are actually deleted, then re-verify from a clean clone.

$ MIRROR=/run/incident/repo-mirror.git
cd "$MIRROR"

# Without --mirror or --prune, refs deleted by the rewrite survive on
# the server and the value stays reachable through them.
git push --mirror origin

# Verify from a clone taken after the push, not from the mirror.
rm -rf /run/incident/verify.git
git clone --mirror https://git.example.com/platform/service.git \
  /run/incident/verify.git
git -C /run/incident/verify.git log --all --full-history \
  -S "$(cat /run/incident/leaked-value)" --format='%H %cI'

An empty result from that final search is the cleanup evidence. It is not confidentiality evidence, and the two must not be confused in the incident record. For the cached views and the review references, open a support case with the platform; the documentation is clear that this is the only route, and equally clear that support may decline where rotation already mitigates the risk.

Step 7 - Close the recurrence gap

Configuration changein the repository that leaked— Route hooks to a tracked directory and confirm the pipeline gate exists.

$ REPO=/srv/work/service
cd "$REPO"

# Route hooks to a tracked directory so every clone runs the same ones.
install -d -m 0755 .githooks
git config core.hooksPath .githooks
git config --get core.hooksPath

# The hook itself calls whichever scanner your organisation has
# standardised on, in its staged-changes mode, at a pinned version.
ls -l .githooks/pre-commit

# The pipeline gate covers everyone who never configured hooks,
# including commits made through the platform web editor.
grep -rn 'secret-scan' .github/workflows || \
  echo 'no pipeline gate configured yet'

Local hooks are advisory: they run only for people who configured them, and never for a commit made through the platform’s web editor. The pipeline gate and the platform’s own push protection are the controls that apply to everyone. Enable both, and note that repository push protection is disabled by default, so nobody turned it off; it was simply never on.

Common pitfalls

Symptom	Cause	Action
A scanner still finds the value after the rewrite	It also appears in a commit message, a tag name or a note, which a path filter does not touch	Repeat the rewrite from the mirror using the text-replacement form rather than a path filter
The value reappears on the server hours later	Someone ran `git pull && git push` from a clone taken before the rewrite	Take a fresh mirror, rewrite again, restrict pushes, and confirm each holder has re-cloned
Old tags still resolve to the original commits	An existing clone does not replace tags it already holds, even with `git fetch --tags`	Instruct holders to delete their local tags before refetching, then verify rather than assume
The rewrite succeeded but review references still show the value	Review namespaces cannot be force-pushed over	Open the platform support case and record the residual risk if it is declined
The replacement works and so does the old value	The provider issued a new credential without disabling the old one	Revoke explicitly, then re-run the outside probe from Step 2
The audit query returns nothing at all	The window was bounded by the author date rather than the push date	Recompute the window from the committer or push timestamp and re-run

Verification

Read the provider’s own interface and confirm the old credential is revoked or absent, rather than inferring it from the command you ran. Probe once more from a host outside the rotation path and record the refusal with its timestamp. Confirm the consumer is healthy on the replacement from its own health endpoint or log stream, not from the tool that deployed the change. Take a fresh mirror clone from the server and confirm that git log --all --full-history -S finds no match for any recorded form of the value, and that git for-each-ref no longer lists the refs the rewrite deleted. Confirm the audit trail shows no use of the old credential after the recorded revocation time. Finally, have someone other than yourself confirm in the repository settings that secret scanning and push protection are enabled, and check that push time, revocation time and audit window reconcile in the incident record.

Rollback

Revocation does not roll back and must not be reversed; if the replacement breaks a consumer, repair the consumer or issue a second replacement rather than reinstating the leaked value. If the rewritten history is wrong, restore from the mirror clone taken during the pre-checks, which is the only complete copy of the pre-rewrite state you control. If the push landed partially and the server now holds a mixture, stop all writes, restore the mirror, and repeat the rewrite in a single pass instead of patching forward. If a contributor has already merged the old history back, take a fresh mirror, rewrite again, and lock pushes until every holder confirms a clean clone. If a consumer cannot move to the replacement inside the window, take it offline and record that decision. If a platform purge breaks an automation that resolved a commit by SHA, repoint the automation rather than asking for the cached view back.