Cache and sharding — the two levers for pipeline duration

Caching and sharding are the two main levers for cutting pipeline wall-clock duration, and they work on different parts of the cost model. Caching reduces the work one job has to do by reusing previously produced artefacts. Sharding splits one job into N parallel jobs so each runs on a smaller slice. They are not alternatives - they compose - but they have different costs and they apply to different workload shapes.

Cache: cut the work within a job

Caching reduces duration by removing redundant computation. A test job that downloads 800 MB of npm packages on every run has a fixed cost floor: even on a fast runner, the download alone takes a minute or two. Caching the ~/.npm directory keyed by the package-lock.json hash replaces the download with a restore that takes seconds.

- name: Cache npm dependencies
  uses: actions/cache@v4
  with:
    path: ~/.npm
    key: npm-${ runner.os }-${ hashFiles('package-lock.json') }
    restore-keys: |
      npm-${ runner.os }-
      npm-

The cache step writes the path at the end of the job (on a cache miss the install step refills it) and restores it on the next job. The exact key is npm-<os>-<sha-of-lockfile>; the restore-keys provide fallback to a near-match. The cache hit makes the install step a no-op: the wheel cache is already populated, and npm ci finishes in seconds instead of minutes.

The workload shape that benefits from caching is any workload with a high proportion of deterministic, expensive setup. npm install, pip install, Maven dependency resolution, Go module download, cargo fetch, Terraform provider plugin download all fit the pattern. The cache step is the answer for all of them.

Sharding: cut the work by splitting the job

Sharding reduces duration by parallelism. A test suite that takes 30 minutes serially can be split into ten shards of three minutes each, run in parallel, with results merged at the end. The wall- clock duration drops from 30 minutes to 3 (plus merge overhead).

flowchart TB
    A["Test suite: 10000 tests, 30 min serial"] --> B["Shard by test path or index"]
    B --> C1["Shard 1: tests 1-1000"]
    B --> C2["Shard 2: tests 1001-2000"]
    B --> C3["Shard 3: tests 2001-3000"]
    B --> C4["Shard ..."]
    B --> C10["Shard 10: tests 9001-10000"]
    C1 --> D["Aggregate results"]
    C2 --> D
    C3 --> D
    C4 --> D
    C10 --> D
    D --> E["Wall-clock: ~3 min + merge"]

The diagram shows the parallelism. Each shard runs on its own runner; the shards do not need to share state during the run because they test disjoint slices. The aggregate step at the end combines the per-shard results into one overall pass/fail signal.

The workload shape that benefits from sharding is any workload with a long serial tail of independent units of work. Test suites are the canonical case. A monorepo build is another: ten packages can be built in parallel and merged into one image. A documentation build is a third: hundreds of Markdown files can be rendered in parallel.

The cost of each lever

Caching and sharding have different costs:

• Cache cost. The cache step uploads on miss and downloads on hit. Both are billable. A cache that is too large to upload efficiently, or that churns on every run, slows the job it was meant to speed up. The cache store also has retention limits (10 GB per repository by default on GitHub Actions).
• Shard cost. Each shard is a separate runner. Ten shards is ten runners’ worth of compute, plus the orchestration cost of starting them, plus the merge step at the end. Sharding a job that already runs in three minutes does not save wall-clock time; it costs ten times the compute.

Choosing the right lever

The right lever depends on where the duration is spent:

• If the duration is in setup (dependency download, toolchain install, provider fetch), cache it.
• If the duration is in execution (test run, build, render), shard it.
• If the duration is in both, do both: cache the setup per shard and shard the execution.

The cost of each lever should be visible in the baseline. A cache hit saves N minutes of install time per job; a shard saves N - 1 of the serial execution time per job (because the longest shard dominates the wall-clock). The lever that saves the most time at the least cost is the one to pull first.

Production discipline

1. Cache the deterministic setup. npm, pip, Maven, Go modules, Terraform providers, cargo crates - all fit the cache pattern.
2. Shard the independent execution. Tests, package builds, documentation renders, and any other workload that can be partitioned into independent units.
3. Compose the two levers. A sharded workflow that does not cache its dependencies pays redundant download cost on every shard.
4. Measure after each change. The baseline is recorded once; the post-change metrics are recorded after each lever is pulled. The lever that produced the largest improvement against the smallest cost is the one to keep.

Cross-course references

• Linux for Production Sysadmins - Part XXXIV (ConfigMgmt) applies the cache pattern to apt/dnf package caches in image builds.
• Ansible for Production Sysadmins - Part XXXVII (RepoArch) applies the sharding pattern to molecule runs across inventory groups.
• Terraform for Production Sysadmins - Parts IX-XII (State) apply the cache pattern to provider plugin downloads and the sharding pattern to per-workspace plans.

Quiz