Replacing content — when a string must change across history

Text replacement is the right tool when a string must change across every commit. The tool is git filter-repo --replace-text; the alternative for large repos is BFG --replace-text.

When text must be replaced

The four text-replacement use cases:

• Email address. A team member changed companies; the old email should not appear in history.
• Credential value. The credential is staying but the value is rotating and the new value must be in history (a public key, a configuration endpoint).
• Name. Married name, preferred name, legal name change. The name should not appear in the old form in history.
• Organisation. Company rebrand, division merge, legal entity change. The old name should not appear in history.

flowchart LR
    A["string must change"] --> B{"single token or per-string?"}
    B -->|"single ***REMOVED***"| C["BFG --replace-text"]
    B -->|"per-string replacement"| D["filter-repo --replace-text"]
    C --> E["reflog expire + gc"]
    D --> E
    E --> F["force-push --force-with-lease"]

The decision is between BFG (fast, single replacement token) and filter-repo (slower, per-string replacement values).

The procedure with filter-repo

The file is plain text, one literal->replacement pair per line:

olduser@oldcorp.com==>newuser@newcorp.com
api.example.com/v1==>api.example.com/v2

The rewrite:

git filter-repo --replace-text expressions.txt

Every blob containing any literal is rewritten with the corresponding replacement. Commit messages are scanned too because commit messages are stored as blobs in the object store. The new blobs have new SHAs; the commit graph follows.

The one-shot pattern for a single replacement uses process substitution:

git filter-repo --replace-text <(echo 'olduser@oldcorp.com==>newuser@newcorp.com')

The pattern is right when there is exactly one literal to replace; the file form is right when there are several.

The procedure with BFG

BFG’s --replace-text replaces every match with the same ***REMOVED*** token:

bfg --replace-text passwords.txt

passwords.txt is one string per line; every match becomes ***REMOVED***. BFG is faster than filter-repo --replace-text for very large repositories because BFG walks the object store directly and skips the commit-graph parsing.

The BFG form is right when the match should be replaced with the same sentinel; the filter-repo form is right when each match has its own replacement value.

Callbacks for path and identity

Path rewrites use --name-callback:

git filter-repo --name-callback 'return name.replace(b"old/", b"new/")'

The callback is a one-line Python expression that takes a bytes path and returns a bytes path (or None to leave unchanged). The mode is right for renaming a path across history (a directory move, a file rename in every commit).

Identity corrections use --mailmap:

git filter-repo --mailmap mailmap.txt

The mailmap.txt is in the standard .mailmap format:

New Name <new@example.com> <old@example.com>
New Name <new@example.com> Old Name <old@example.com>

Every author and committer identity in the history is rewritten to the canonical form. The mode is right when the correction is an identity (name + email), not a string in file contents.

Verification before push

Three checks before the force-push:

git log --all -S'olduser@oldcorp.com'
git log --all --grep='olduser@oldcorp.com'
git rev-list --all --objects | grep -i 'oldcorp'

The first checks blob contents (pickaxe); the second checks commit messages; the third checks every object reachable from any ref. All three should print nothing before the force-push.

Edge cases

• The literal appears in many files. Use --replace-text; the file form is more maintainable than per-file callbacks.
• The replacement is sensitive. Use BFG --replace-text so the replacement is the same ***REMOVED*** token and no new credential appears in history.
• The literal is a regex. filter-repo --replace-text is literal-only; for regex substitution use --blob-callback with Python’s re module.

Production discipline

1. The expressions file is the audit trail. The file lists every literal that was replaced; preserve the file for the security audit.
2. Verify with three checks before pushing. Pickaxe for blob contents, grep for commit messages, rev-list for every reachable object.
3. --force-with-lease always. The safety catch against clobbering commits that landed during the rewrite window.

Cross-course references

• Linux for Production Sysadmins - Part XXXIV (ConfigMgmt) covers the configuration-file rewrites that mirror the text replacement.
• Ansible for Production Sysadmins - Part XXXVII (RepoArch) covers the inventory rewrites that benefit from the mailmap mode.
• Terraform for Production Sysadmins - Parts IX-XII (State) cover the state-file backend rewrites that may follow a history rewrite.

Quiz