Availability, Latency, Throughput, Errors, Saturation

A service has five operational dimensions: availability, latency, throughput, errors, and saturation. The platform’s SLI work is to construct a measurement for each dimension and decide which to SLO against.

Different methodologies (USE, RED) pick different subsets. Each methodology is useful for a different class of system. The course introduces them here and returns to each in production depth later.

Availability

Availability is the fraction of requests that succeed. The canonical SLI:

SLI = successful_requests / total_requests

A request is “successful” if it returns 2xx or 3xx (depending on the service’s definition) within a reasonable time. A request that hangs forever is not successful — the timeout must be enforced to convert hangs into failures.

A common implementation:

SLI = sum(rate(http_requests_total{status!~"5xx"}[5m]))
    / sum(rate(http_requests_total[5m]))

For a strict uptime definition (the service is available when it serves any 2xx in a window), availability is computed differently:

availability = fraction_of_1m_windows_with_at_least_one_2xx

The SLO target is usually 99.9% or 99.99% for a critical service. The target is determined by what the user expects and what the team can deliver; “five nines” (99.999%) is rare and expensive.

Latency

Latency is the time between request and response. The canonical SLI:

SLI = fraction_of_requests_under_threshold / total_requests

A 99th-percentile latency SLO is common:

SLI = sum(rate(http_request_duration_seconds_bucket{le="0.5"}[5m]))
    / sum(rate(http_request_duration_seconds_count[5m]))

The threshold is the user’s expectation, not an internal target. An API whose user expects <500#60;500 ms should SLO at <500#60;500 ms. An API whose user expects <5 s#60;5 s should SLO at <5 s#60;5 s.

Latency is sensitive to outliers. A streaming 99th percentile smooths over sudden spikes; an exact 99th percentile is noisy. The histogram_quantile() function in Prometheus works on histograms and is the canonical implementation. Native histograms (Prometheus 2.50+) compute exact quantiles with much less overhead.

Throughput

Throughput is the rate of work. A request rate:

rate = sum(rate(http_requests_total[5m]))

Throughput is not a user-visible SLI — the user does not care how fast requests arrive. But it is a key operational metric: throughput changes correlate with capacity changes, and a sudden drop in throughput is a strong signal that the service is failing.

The next failure shape:

SLI of /checkout has been 99.9% for weeks
throughput has dropped from 1000/s to 200/s

The SLO did not fire. The service is failing for one in five users but the failing requests are a small fraction in absolute numbers because the overall request rate has dropped.

Errors

Errors are explicit failure signals. The canonical SLI:

error_rate = sum(rate(http_requests_total{status=~"5xx"}[5m]))
            / sum(rate(http_requests_total[5m]))

Errors are also useful as raw counts:

5xx_count = sum(rate(http_requests_total{status=~"5xx"}[5m]))

A sudden increase in errors is the highest-confidence symptom of a service-wide failure. A stable error rate but elevated latency can mean many things; an elevated error rate is usually failure-shaped.

Saturation

Saturation is a measure of how “full” the service is. Examples:

• CPU utilisation on a host that is not horizontal-scale.
• Memory pressure (memory used / memory available).
• Connection pool saturation (connections in use / connections available).
• Queue depth (queue size / queue capacity).

Saturation is not a user-visible SLI — the user does not see CPU. But saturation predicts failure: a service approaching saturation usually fails soon after. Saturation alerts are leading indicators; error rate alerts are lagging.

How the dimensions interact

The five dimensions are not independent. They interact:

• Throughput × latency = concurrency. A service handling 1000 req/s with average latency 100 ms has concurrency of ~100. Saturation at 1000 concurrent requests in a 1000-request pool means queues grow and latency spikes.
• Errors correlate with saturation. A service whose connection pool saturates returns 503s. The error rate alert is downstream of the saturation alert.
• Latency correlates with throughput. As throughput grows, latency grows non-linearly. A service with a latency SLO is fundamentally constrained by its capacity SLO.
• Availability is the trailing form of errors. Availability over a window is the integrated form of error rate.

Choosing the right SLI pair

The canonical pair for an HTTP API is:

• Availability SLO. 99.9% of requests non-5xx over 30 days.
• Latency SLO. 99% of requests <500#60;500 ms over 30 days.

Both are user-visible. Both are measured from the same histogram + counter pair. Both answer questions the user asks.

A team that does not pick a pair picks an SLO mosaic: dozens of partially-relevant metrics, none of them anchor.

Production guidance

• Compute the five dimensions from a small set of counters and histograms. Most services instrument requests_total and request_duration_seconds. The dimensions are computed from these primitives.
• Choose user-visible SLIs (availability, latency) over internal ones (saturation, CPU). User-visible SLIs are what alerts; internal metrics are what diagnoses.
• Use burn-rate alerting on the user-visible SLIs (Part XXII). The internal dimensions are reactive alerts.
• Make sure the histograms and counters share a label set. Divergent label sets make PromQL aggregation painful.

Verification

You should be able to answer:

• What are the five operational dimensions?
• Which are user-visible (SLI-eligible) and which are internal (saturation)?
• Construct PromQL for each dimension.
• How do the dimensions interact?

Quiz