CPU capacity — quota, throttling, and the utilisation you can actually use

Memory capacity is a question about a ceiling: exceed it and you die. CPU capacity is a question about a queue: exceed it and you wait. The two failure modes look nothing alike, and the mistake operators make is planning CPU as though it were memory.

--cpus 0.5 does not give a container half a CPU. It gives it 50 ms of CPU time in every 100 ms window, and when those 50 ms are used up the container stops running until the window rolls over — no matter how idle the host is.

What --cpus compiles to

CID=web
PID=$(docker inspect --format '{{.State.Pid}}' "$CID")
CG=$(awk -F: '{print $3}' "/proc/$PID/cgroup" | head -1)
cat "/sys/fs/cgroup$CG/cpu.max"

50000 100000

The quota is a hard ceiling. The weight is a tie-breaker. They solve different problems and you generally want exactly one of them: quota when you are protecting other tenants from this container, weight when you are ranking containers that all legitimately want the whole host.

The sizing formula

CPU demand is throughput times cost per unit of work:

cpu_seconds_per_second = rps x cpu_seconds_per_request
quota_cores            = cpu_seconds_per_second / target_utilisation

target_utilisation is where the arithmetic stops being obvious. It is not 1.0, and it is not “80% because everyone says 80%”. Queueing gives the reason: for a server with random arrivals, the mean wait scales as U / (1 - U). At 50% utilisation you wait as long as one service time. At 80% you wait four times a service time. At 90%, nine.

Worked example. An API serves 400 requests per second at peak, and a profile shows 3.5 ms of CPU per request:

cpu_seconds_per_second = 400 x 0.0035     = 1.4 core-seconds/second
quota_cores (at 0.65)  = 1.4 / 0.65       = 2.15 cores
                                          -> --cpus 2.25

Round up, not down: rounding a CPU quota down does not degrade gracefully, it throttles.

CID=api
PID=$(docker inspect --format '{{.State.Pid}}' "$CID")
CG=$(awk -F: '{print $3}' "/proc/$PID/cgroup" | head -1)
A=$(awk '/^usage_usec/ {print $2}' "/sys/fs/cgroup$CG/cpu.stat")
sleep 60
B=$(awk '/^usage_usec/ {print $2}' "/sys/fs/cgroup$CG/cpu.stat")
echo "cpu-seconds used in 60s: $(( (B - A) / 1000000 ))"

cpu-seconds used in 60s: 84

84 core-seconds in 60 wall-seconds is 1.4 cores of steady demand, which is the figure the formula wanted. Divide by the request count over the same minute for the per-request cost.

Throttling: the signal that average CPU hides

This is the single most useful thing in the lesson. A container can sit at 20% average CPU and still be throttled hard, because the average is taken over minutes and the quota is enforced over 100 ms. A request that needs 200 ms of CPU on a --cpus 1.0 quota cannot complete inside one period. It runs 100 ms, stalls for the remainder of the window, runs again. The user sees 200+ ms of latency for 100 ms of work, and the CPU graph shows a comfortable 20%.

for c in $(docker ps -q); do
  PID=$(docker inspect --format '{{.State.Pid}}' "$c")
  CG=$(awk -F: '{print $3}' "/proc/$PID/cgroup" | head -1)
  NAME=$(docker inspect --format '{{.Name}}' "$c")
  awk -v n="$NAME" '/^nr_periods/{p=$2} /^nr_throttled/{t=$2}
    END {if (p > 0) printf "%-28s %8d periods %6d throttled %6.3f%%\n", n, p, t, 100*t/p}' \
    "/sys/fs/cgroup$CG/cpu.stat"
done

/collector                    228238 periods      2 throttled  0.001%
/dashboards                  3411280 periods      9 throttled  0.000%
/metrics                      750729 periods      0 throttled  0.000%
/logs                        2407614 periods      3 throttled  0.000%
/traces                       364877 periods      7 throttled  0.002%
/db                           208131 periods      0 throttled  0.000%
/cache                       2488049 periods      1 throttled  0.000%

That is a healthy fleet: every container is under 0.01% throttled periods. Read the ratio, not the raw count — a container that has been up for a month has millions of periods, so nine throttle events is noise while nine hundred thousand is a production incident.

Rules of thumb for nr_throttled / nr_periods:

• Below 0.1% — ignore it.
• 0.1% to 1% — real but tolerable for async work; investigate for anything user-facing.
• Above 1% — the quota is wrong for the workload’s burst shape, and the p99 latency chart will already show it.

cpu.pressure — the queueing you cannot see in a percentage

CID=api
PID=$(docker inspect --format '{{.State.Pid}}' "$CID")
CG=$(awk -F: '{print $3}' "/proc/$PID/cgroup" | head -1)
cat "/sys/fs/cgroup$CG/cpu.pressure"

some avg10=0.00 avg60=0.00 avg300=0.00 total=1116773
full avg10=0.00 avg60=0.00 avg300=0.00 total=1102651

some avg300 above about 10 means tasks in this container spent more than a tenth of the last five minutes waiting for CPU that was not available. Unlike a utilisation percentage, that number is already in the units you care about: delay.

Host budgeting: CPU is not memory

Memory oversubscription is a bet you lose all at once. CPU oversubscription is a bet you lose gradually, which makes it a reasonable bet.

sum(quota_cores)  <=  cores x oversubscription_ratio

SUM=$(docker ps -q | xargs -r docker inspect --format '{{.HostConfig.NanoCpus}}' | awk '{s+=$1} END {print s+0}')
CORES=$(nproc)
awk -v s="$SUM" -v c="$CORES" 'BEGIN {
  printf "committed quota: %.2f cores\n", s/1e9
  printf "host cores:      %d\n", c
  printf "ratio:           %.2f\n", (s/1e9)/c
}'

committed quota: 8.00 cores
host cores:      12
ratio:           0.67

As with memory, a container without --cpus reports 0 and is counted as nothing while being able to take everything. Find those first.

Sanity check