SLIs, SLOs, and Error Budgets

SLI, SLO, and error budget are the production discipline that turns telemetry into a contract. Without them, dashboards describe internal state and alerting is ad hoc. With them, the team has a single question to answer: “is the error budget consumed faster than the window allows?”

This lesson defines each term precisely. It does not turn the course into an SRE mathematics course — it grounds the SLI/SLO discipline as the prerequisite for everything else.

What an SLI is

A Service Level Indicator is a measured quantity. It answers a question about the service:

• ‘What fraction of /checkout HTTP requests succeed in <2 s?’
• “What fraction of /api/orders HTTP requests return 5xx?”
• “What is the 99th-percentile latency of /api/recommendations?”

Three properties make a useful SLI:

1. User-visible. The SLI measures what the user experiences, not what the service exposes. “Process goroutine count” is not user-visible. “API success rate” is.
2. Measurable. The SLI can be computed from existing telemetry. A counter divided by a counter, a histogram’s quantile.
3. Reproducible. Different operators reading the same SLI should arrive at the same number. “It feels slow” is not reproducible. “p99 latency of /api/orders measured over 5 minutes” is.

The SLI is the measurement. It is not the target. Many SLIs exist; the team picks one (or two) that best describes user experience.

What an SLO is

A Service Level Objective is a target over an SLI:

• “99% of /checkout HTTP requests succeed in <2 s, measured over a 30-day window.”
• “Less than 0.1% of /api/orders HTTP requests return 5xx, measured over a 30-day window.”

The SLO is the contract. It states the level of service the team commits to. Crossing the SLO is a violation of the contract.

The SLO has components:

1. The SLI. (What is measured.)
2. The target. (What value the SLI must reach.)
3. The window. (Over what period the SLI is averaged — usually 30 days, sometimes 7.)

A service has one SLO per user-visible behavior — usually one for availability, one for latency. A team tracking eight separate SLOs is over-engineered and is tracking metrics that do not describe user experience.

What an error budget is

The error budget is the unreliability the SLO permits:

• SLO: 99% of /checkout requests succeed in <2 s over 30 days.
• Window: 30 days = 2,592,000 s. If the service receives 1,000 requests / s, the request count over 30 days is 2,592,000,000. A 1% error budget allows 25,920,000 of those requests to fail.
• Burn rate at 1×: 25,920,000 errors / 2,592,000 s = 10,000 errors per second (sustained).

The error budget is the lever that ties engineering work to operational discipline. When the budget is intact, the team takes feature work. When the budget is consumed (e.g. a recent incident consumed 5% of the budget), feature work pauses and reliability work takes priority.

What makes a good SLO

A good SLO is:

• User-visible. It describes user experience, not internal resource.
• Measurable. The team can compute it from existing telemetry.
• Few. One availability SLO, one latency SLO. Maybe an availability/latency SLO for a critical endpoint.
• Audited. The SLO is verified: the metric is correct, the data is good, the formula matches the intent.

A bad SLO is:

• An internal resource as the user metric. (“CPU above 90% is bad” is not an SLO. The user does not see CPU.)
• Too many. Tracking eight SLOs per service dilutes them all.
• Unmeasured. A target without a measurement is a wish.
• Unowned. An SLO without an owner is a wish.

Choosing the right SLI

The SLI choices for a typical web service:

A common SLO design for an HTTP API:

• Availability SLO: 99.9% of requests return non-5xx over a 30-day window.
• Latency SLO: 99% of requests return in <500 ms over a 30-day window.

These two are the canonical pair. A team that owns more than a few endpoints chooses a similar pair per endpoint.

Constructing an SLO in Prometheus

For an HTTP API with a request_duration_seconds histogram and a http_requests_total counter:

SLI: successful_requests / total_requests
  = sum(rate(http_requests_total{status!~"5xx"}[30d]))
    /
    sum(rate(http_requests_total[30d]))

SLI: fraction of requests < 500ms
  = sum(rate(http_requests_duration_seconds_bucket{le="0.5"}[30d]))
    /
    sum(rate(http_requests_duration_seconds_count[30d]))

SLO target: SLI >= 0.999

Error budget: (1 - SLI) × total_requests over 30d

Burn rate (errors per second):
  sum(rate(http_requests_total{status=~"5xx"}[5m]))

Burn rate is the bridge to alerting. The course returns to burn- rate alerting in Part XXII.

Failure modes

• The over-specified SLO. A team tracks a 99.99% availability SLO because the vendor advertises it. The service has 1 request per hour. The 0.01% budget allows one request every 10,000 hours — over a year, but the team’s actual reliability is much worse. The SLO is misleading. Pick the lowest target whose violation matters.
• The under-specified SLO. A team tracks a 90% availability SLO because that is comfortable. The production user impact is much better; the SLO does not capture user experience. The SLO is too lax.
• The unmeasured SLO. A team picks a target but does not verify the measurement. The SLO is a number on a dashboard; nobody believes it. The SLO does not produce discipline.
• The SLO with no alerting. A team computes the SLO every week. They do not alert on it. The SLO is reported post-incident. The SLO is too late.

A useful SLO is auditable, alerts on the right threshold, and is owned by a team.

Verification

You should be able to answer:

• What is the difference between an SLI and an SLO?
• How does an error budget convert a SLO into prioritisation?
• What makes a good SLO?
• What failure modes appear in SLO designs?

Quiz