Ephemeral runner sizing — per-job cost and the speed-versus-money trade-off

Runner sizing is not about picking the cheapest instance type. The per-job cost of an ephemeral runner has three components - compute, overhead, and image-pull - and the shape that wins on compute can lose on overhead. The right question is not “what is the cheapest runner” but “what is the cheapest unit of useful work”.

Anatomy of a per-job cost

An ephemeral runner job pays for:

flowchart LR
  J["Job"] --> C["Compute\nvCPU-min, GB-min"]
  J --> O["Overhead\nstartup, agent handshake,\nstep coordination"]
  J --> I["Image pull\ncold-cache, registry RTT,\nlayer verification"]
  J --> S["Storage\nworkspace I/O,\nartifact upload"]

• Compute. The vCPU × minutes (or per-minute equivalent) the runner charges for. On GitHub-hosted this is the per-minute rate of the chosen runner image.
• Overhead. Agent startup, runner registration, job scheduling. On a self-hosted runner this is small (seconds); on a cold-started Kubernetes pod it can be 30-60 seconds.
• Image pull. Pulling the runner image and the build environment. A 2 GB image over a fast link takes 8 seconds; over a slow link or a large image it can take minutes.
• Storage. Workspace I/O during the job, plus artifact upload at the end.

A 30-second job on a runner with 60 seconds of overhead pays 3x as much per useful second as a 30-second job on a runner with 10 seconds of overhead.

The right-sizing decision

Three patterns drive the decision:

• CPU-bound jobs. Compiler builds, image builds, test suites that parallelise across cores. Larger runners dominate; the cost-per-job is roughly constant across shapes, and the speed win compounds across the day.
• I/O-bound jobs. Tests that wait on network calls, jobs that upload large artifacts. Larger runners do not help; faster disks and faster network do.
• Single-threaded jobs. Linters, format checks, manifest validators. Larger runners waste money; a small runner is right-sized for the workload.

The pool should reflect the workload mix. A team that runs 60% CPU-bound jobs and 40% I/O-bound jobs should consider two pools with different runner shapes, not one pool of a medium-sized runner.

The cold-start budget

The biggest “invisible” cost on ephemeral runners is cold-start latency:

ARC_NS=actions-system
kubectl get hpa -n "$ARC_NS"

This shows the horizontal pod autoscaler that backstops the runner pods. Cold-start latency is the gap between “workflow_job queued” and “runner pod ready”. On ARC with minRunners: 0 the gap is dominated by:

• Pod scheduling (1-5 seconds on a healthy cluster).
• Container image pull (8-60 seconds depending on image size).
• Runner agent startup (5-10 seconds).
• Job checkout (variable, often 10-30 seconds).

For a 10-minute job, a 60-second cold start is 10% overhead

• tolerable. For a 30-second job, a 60-second cold start is 100% overhead - the runner costs twice what the job does. The fix is minRunners: 1 for latency-sensitive pools, or a pre-warmed image cache.

Cost per useful minute

The right metric is cost per useful minute of work, not cost per runner-minute:

cost_per_useful_minute = (overhead + job_duration) × per_min_rate / job_duration

OVERHEAD_MIN=1
JOB_MIN=5
RATE_PER_MIN=0.008
COST_PER_USEFUL=$(awk -v o="$OVERHEAD_MIN" -v j="$JOB_MIN" -v r="$RATE_PER_MIN" \
  'BEGIN { printf "%.5f", (o+j)*r/j }')
echo "Cost per useful minute: $COST_PER_USEFUL"

A small runner with high overhead can cost more per useful minute than a large runner with low overhead. The comparison must be at the workload level, not the catalogue level.

Production discipline

1. Measure per-job cost, not per-minute rate. The minute rate is the input; the job cost is the outcome.
2. Match the runner shape to the bottleneck. Larger runners help only for CPU-bound work.
3. Set minRunners based on cold-start budget, not on intuition. A 10% cold-start is acceptable; 100% is not.
4. Right-size the image. A 4 GB runner image doubles cold-start time on slow links; trim to what the job actually uses.
5. Compare pools at the workload level. A 30-runner pool of small runners can beat a 10-runner pool of large runners for I/O-bound work.

Cross-course references

• Git, CI/CD & GitOps - Parts XLVII (Cache) and LIII (ImageSupply) cover the image-pull cost component.
• FinOps for Production Sysadmins - Part VIII (Rightsizing) covers the same trade-off for general compute.
• Kubernetes for Production Sysadmins - Part XXII (HPAMath) covers the pod-startup component.

Quiz