Build cache maintenance — the quiet largest consumer · Docker & Containers

On a host that builds images, build cache is usually the largest single consumer of disk and the last one anyone looks at. It does not appear in docker images, it has no tags, and its entries are opaque hashes. The 7.4 GB in the Build Cache row of the previous lesson was larger than every container writable layer on the host by four orders of magnitude.

It is also the only Docker object with a real garbage collector, which makes it the easiest of the four categories to stop worrying about.

Measuring it

docker system df gives you the total. docker buildx du gives you the records.

Read-only / Safebuildx du— Per-record build cache usage, oldest access first.

$ docker buildx du

ID                           RECLAIMABLE   SIZE       LAST ACCESSED
j5j2jprjqbdgiobhyqqukak41    true          248B*      10 days ago
50w5fx23527c0ex1kai5m4nss    true          405B*      6 days ago
7zxq95ogag1jtq3tc40zfqgcz    true          957B*      6 days ago
j66nkr4hb6t12vmyoz92ple0c    true          553.1MB    6 days ago
Shared:         2.322GB
Private:        5.124GB
Reclaimable:    7.446GB
Total:          7.446GB

Illustrative output

Four numbers at the bottom, and they answer different questions:

Shared — cache blobs that are also referenced by an image on this host. Pruning the cache record does not free these, because the image still holds the layer.
Private — blobs held only by the cache. This is what a prune actually frees.
Reclaimable — the total of records marked RECLAIMABLE true.
Total — everything, including records currently pinned by a running build.

The asterisk on some sizes marks a shared record. When you are estimating how much a prune will return, Private is the number, not Reclaimable.

Read-only / Safeverbose du— Show what each record actually is.

docker buildx du --verbose

The verbose form adds the Dockerfile step or mount that produced each record. That is how you find out that 5 GB of your cache is one RUN npm ci layer retained across forty builds of a Node image.

Why it grows without bound

What BuildKit keeps and why

BuildKit’s cache is content-addressed and keyed on the full build graph. Every distinct combination of a build step and its inputs produces a distinct cache record, and BuildKit keeps records for steps it might reuse.

The consequence: cache size scales with the number of distinct builds, not with the number of images. A CI runner that builds the same Dockerfile fifty times a day against fifty different commits produces fifty distinct COPY . . results and fifty distinct downstream steps, all retained. The image count stays at one. The cache grows every build.

Three kinds of record accumulate particularly fast:

RUN --mount=type=cache directories — package manager caches for npm, pip, apt, Go modules. These are deliberately persistent and can be gigabytes on their own.
Multi-stage build intermediates — every stage produces cache records, including the fat build stage you discarded from the final image.
Frontend and internal images — the Dockerfile frontend BuildKit pulls to parse your file. Small, but they are what -a on prune targets.

Pruning with a bound

The wholesale form is the one everyone runs and the one you should almost never use:

Destructivewholesale— Removes all reclaimable cache. The next build has nothing to reuse.

docker builder prune -a -f

-a on build cache means “include internal and frontend records too” — it is not the same widening as -a on images, but the practical effect is the same: the next build starts cold.

The bounded forms are better:

Destructivebounded prunes— Keep the cache useful while capping its size.

# Drop records not touched in a week
docker builder prune --filter 'until=168h' -f

# Keep a floor of cache and prune down to it
docker builder prune --reserved-space 10GB -f

# Prune until the cache is under a ceiling
docker builder prune --max-used-space 20GB -f

# Prune until the disk has this much free
docker builder prune --min-free-space 30GB -f

Configuring garbage collection so you never prune by hand

This is the part worth doing once. BuildKit inside dockerd reads its GC policy from daemon.json:

Configuration changedaemon.json— Cap the build cache at the daemon level.

{
"builder": {
  "gc": {
    "enabled": true,
    "defaultKeepStorage": "20GB"
  }
}
}

With enabled: true and a keep-storage figure, BuildKit runs its own collection in the background and holds the cache near that size. The default policy behind that single number is a four-tier sweep, in order:

Ephemeral cache (internal and frontend records) older than 48 hours.
Any cache older than 60 days.
Unshared cache over the size limit.
Any cache over the size limit.

That ordering matters: BuildKit gives up the cheap-to-rebuild records first and only touches shared, recently used records when it has no other choice.

For a builder with its own buildkitd.toml — a docker-container driver builder rather than the default in-daemon one — the equivalent keys live under [worker.oci] and use reservedSpace, maxUsedSpace and minFreeSpace.

Cache loss versus cache invalidation

When builds suddenly take ten times longer, the cause is one of two things and the remedies are opposite.

Symptom	Cause	Fix
Every step re-runs, including ones whose inputs did not change	Cache loss — the records were pruned or GC removed them	Re-warm the cache; raise the GC cap
Steps re-run from a specific line onwards, consistently	Cache invalidation — an input to that step changed	Reorder the Dockerfile; this is a build design problem
First build after a deploy is slow, subsequent ones fast	Normal cold cache	Nothing

Read-only / Safediagnose— Check whether the cache still holds what the build needs.

docker buildx du --verbose | head -40
docker system df

If Total collapsed since yesterday, someone pruned. If it is unchanged and builds are still slow, the Dockerfile changed. The build caching lesson in the build part of this course covers the second case; this lesson is only about the first.