Artifact and layer caching — Docker layer cache and build cache

Container builds have their own caching model, distinct from actions/cache@v4 for npm or pip. The Docker layer cache reuses unchanged layers across builds. The BuildKit build cache reuses intermediate artefacts (compilation results, intermediate images) by content hash. Both cut wall-clock duration on the build step itself; neither replaces the dependency cache for the package manager inside the image.

Layer cache versus build cache

The two caches answer different questions:

• Layer cache reuses layers that did not change between builds. A RUN apt-get install line whose input did not change is reused; its layer is not re-executed. The layer cache is implicit and free: it is whatever docker build has on the runner’s local storage.
• Build cache (BuildKit, Buildx) reuses intermediate artefacts by content hash. The output of a multi-stage build’s RUN go build step is reused on the next build if the inputs (Go source, go.mod, the parent stage) did not change. The build cache is explicit: it is configured via --cache-from and --cache-to.

flowchart TB
    A["Dockerfile + context"] --> B{"Layer changed?"}
    B -->|no| C["Reuse layer"]
    B -->|yes| D["Rebuild layer"]
    D --> E{"Build cache hit?"}
    E -->|yes| F["Restore intermediate artefact"]
    E -->|no| G["Run step from scratch"]
    C --> H["Image assembled"]
    F --> H
    G --> H
    H --> I["Image pushed to registry"]

The diagram shows the resolution order. The layer cache is checked first because it is the cheapest reuse path. The build cache is checked second, when a layer must be rebuilt but its intermediate artefact is unchanged. A complete miss rebuilds the step from scratch.

The registry-backed cache

The build cache can live in three places:

• Inline (type=inline) is stored in the image itself. The cache travels with the image. The next build pulls the image and extracts the cache. Simple but limited to single-architecture builds.
• Registry (type=registry) is stored as a separate image in a registry. The cache is durable across runners and across repositories. The standard choice for CI.
• Local (type=local) is stored on the runner’s filesystem. The cache is fast but per-runner: a build on runner A does not see the cache from runner B.

- name: Build and push
  uses: docker/build-push-action@v5
  with:
    context: .
    push: true
    tags: ghcr.io/ORG/IMAGE:${ github.sha }
    cache-from: type=registry,ref=ghcr.io/ORG/IMAGE:buildcache
    cache-to: type=registry,ref=ghcr.io/ORG/IMAGE:buildcache,mode=max

The cache-from pulls the registry-backed cache; the cache-to writes the new cache entries on push. mode=max includes all intermediate layers, not just the final image layers; the standard choice for CI where the entire layer graph is reusable.

The cache-key failure mode

A cache key that does not match the Dockerfile change pattern produces two failure modes:

• Too few hits. A key that includes only the final tag does not distinguish between builds with different intermediate steps. A change to a RUN line deep in the Dockerfile produces a cache key identical to the previous build’s key; the cache returns the previous intermediate artefacts, which are now stale, and the build silently uses them. This is a stale-hit failure mode: the cache says it hit, but the artefact is wrong.
• Too many misses. A key that includes the entire Dockerfile hash changes on every whitespace edit. A comment change in the Dockerfile invalidates every cache entry, even though no functional layer changed. This is a fragile-key failure mode: the cache thrashes on cosmetic edits.

The right cache key for the registry-backed cache is one that matches the inputs to the build step. For most workflows, the default key produced by cache-from: type=registry (which keys by the chain ID of each layer) is correct: BuildKit computes the chain ID from the inputs to the layer, and a whitespace change that does not affect the layer’s input hash produces a cache hit.

Composing with actions/cache@v4

The Docker build cache and the actions/cache@v4 dependency cache compose. A workflow that builds an image can cache the application dependencies (npm, pip) with actions/cache@v4 and cache the image build with the registry-backed build cache. The two caches answer different questions: the first cuts the npm ci step inside the image; the second cuts the image build itself. A workflow that has only one of the two caches is missing half the saving.

- name: Cache npm dependencies
  uses: actions/cache@v4
  with:
    path: ~/.npm
    key: npm-${ runner.os }-${ hashFiles('package-lock.json') }
    restore-keys: npm-${ runner.os }-
- name: Build and push
  uses: docker/build-push-action@v5
  with:
    context: .
    push: true
    tags: ghcr.io/ORG/IMAGE:${ github.sha }
    cache-from: type=registry,ref=ghcr.io/ORG/IMAGE:buildcache
    cache-to: type=registry,ref=ghcr.io/ORG/IMAGE:buildcache,mode=max

Production discipline

1. Use the registry-backed cache for CI on ephemeral runners. The local cache is per-runner and is destroyed with the runner.
2. Use mode=max for the registry cache. The intermediate layers are reusable on the next build; only the final layer changes per commit.
3. Compose with actions/cache@v4 for application dependencies. The build cache does not cache the package manager’s wheel store.
4. Monitor the cache-hit rate on the build step. A drop in the build-step cache-hit rate is a signal that the Dockerfile has changed in a way that invalidates many entries.

Cross-course references

• Linux for Production Sysadmins - Part XXXIV (ConfigMgmt) applies the same content-hash cache pattern to base-image layer caching in golden images.
• Ansible for Production Sysadmins - Part XXXVII (RepoArch) applies the pattern to molecule image builds.
• Terraform for Production Sysadmins - Parts IX-XII (State) apply the pattern to provider-plugin image caches.

Quiz