BFG Repo-Cleaner — the fast credential scrubber

BFG Repo-Cleaner is a Java tool focused on the credential-scrub use case. It is faster than filter-repo because it walks the object store directly and lets the commit graph follow; it cannot run callbacks or mailmap.

Why BFG exists

For very large repositories the general-purpose parsing in filter-repo is the slow part. The credential scrub itself is just a string match against blob contents. BFG drops the commit-graph parsing and optimises for the string-match path:

flowchart LR
    A["rewrite needed"] --> B{"general or blob-only?"}
    B -->|"general (callbacks, mailmap)"| C["filter-repo"]
    B -->|"blob-only, large repo"| D["BFG"]
    C --> E["reflog expire + gc"]
    D --> E
    E --> F["force-push --force-with-lease"]

Install

BFG is a single Java JAR (JRE 8+):

curl -O https://repo1.maven.org/maven2/com/madgp/bfg/1.14.0/bfg-1.14.0.jar
mv bfg-1.14.0.jar /usr/local/bin/bfg.jar
alias bfg='java -jar /usr/local/bin/bfg.jar'

—replace-text

The primary BFG use case. The file is a list of strings to replace with ***REMOVED***:

bfg --replace-text passwords.txt

Every blob is scanned; every match becomes ***REMOVED***; the commit graph follows automatically. Right when the credential is scattered across many files. BFG does not support the literal->replacement syntax that filter-repo does; every match becomes the same token.

—delete-files

Remove every blob whose path matches a glob:

bfg --delete-files '*.{env,key,pem}'

Right when the sensitive file has a recognisable extension or naming pattern.

—strip-blobs-bigger-than

Remove every blob over a size threshold (the accidentally- committed large file: a database dump, node_modules, a test fixture):

bfg --strip-blobs-bigger-than 100M

The threshold is a number with a unit suffix (K, M, G). Right for hygiene (repository size, clone time) rather than credentials. For credential-bearing large files, use --replace-text or --delete-files so the audit log names the specific object removed.

—no-blob-protection

BFG by default leaves the most recent commit’s blobs alone (“blob protection”). --no-blob-protection rewrites every blob including those in the latest commit. Right when the credential is in the latest commit and the team is certain the rewrite should reach it; wrong when the latest commit might be the one the engineer forgot to scrub.

When to use BFG versus filter-repo

• BFG: large repo, blob-only rewrite, common credential patterns, large files. Fastest path for bulk text and bulk blobs.
• filter-repo: callback logic, mailmap, conditional substitution, identity correction.

Both produce the same shape of rewritten history; both require the same reflog-and-gc cleanup; both require the same --force-with-lease push.

Production discipline

1. BFG runs on a bare clone; filter-repo on a regular clone. The difference is the working tree.
2. Same safety contract as filter-repo: fresh clone, reflog expire, gc, --force-with-lease.
3. Verify before pushing. git log -S'<literal>' should print nothing; git log --diff-filter=D --name-only | grep <pattern> should list only the removed paths.

Cross-course references

• Linux for Production Sysadmins - Part XXXIV (ConfigMgmt) covers the file-system hygiene that --strip-blobs-bigger-than mirrors.
• Ansible for Production Sysadmins - Part XXXVII (RepoArch) covers the repository layout that benefits from the bulk-blob rewrite.
• Terraform for Production Sysadmins - Parts IX-XII (State) cover the state-file size that --strip-blobs-bigger-than is sometimes used against.

Quiz