Impact assessment - measuring what is broken for whom

linux-incident-roles-and-communication gives the severity table and the rule that severity comes from observable impact. This lesson is about the step before that: establishing what the impact actually is, quickly and from evidence, so the severity you declare is a reading rather than a guess.

Severity is a decision. Impact is a measurement. Getting them the wrong way round — deciding the severity and then describing the impact to match — is how an incident acquires the wrong response in its first five minutes and keeps it for an hour.

Impact has four dimensions

Describing impact as a single adjective (“bad”, “major”) loses the information the response needs. Four questions, each with a different consequence:

Reversibility is the one that gets left out and the one that overrides the others. An incident affecting 2% of users is a minor availability problem and a major incident if that 2% is silently having writes dropped. The question is not “how bad does this look” but “what will still be wrong after we fix it”.

Measure from the edge, not from the component

The most common assessment error is to measure impact where the fault is. The failing component tells you about itself; it does not tell you what users experience, and the relationship between the two is rarely obvious.

• A failed replica in a three-node set is 100% broken and 0% impact.
• A cache node lost is 0% broken by every health check and substantial impact, because the database behind it is now taking the full read load.
• A load balancer removing an unhealthy backend is working correctly and may be the reason latency doubled.

Ask the question at the service boundary:

$ curl -sS -o /dev/null -w 'code=%{http_code} total=%{time_total}s\n' https://shop.example.com/checkout

code=503 total=0.184s

One probe is an anecdote. A short loop turns it into a rate, which is what scope and depth are actually made of:

# Substitute your own values before running:
URL=https://shop.example.com/checkout

# 30 samples over 30 seconds, counted by status code
for i in $(seq 1 30); do
  curl -sS -o /dev/null -w '%{http_code}\n' --max-time 5 "$URL" || echo "timeout"
  sleep 1
done | sort | uniq -c | sort -rn

     22 503
      7 200
      1 timeout

That is a measurement you can declare from: roughly three requests in four are failing, and the failure is fast rather than a hang. If the ratio had been 1 in 30, the incident is a different one with a different severity.

Where the platform has proper telemetry, use it — a request-rate and error-rate graph at the edge answers this better than any loop. The loop is what you run when the dashboard is part of the outage, which happens more often than anyone plans for.

For a host-local service, count the errors the application is actually emitting rather than reading its status:

# Substitute your own values before running:
UNIT=checkout

# Error lines per minute over the last 15 minutes
journalctl -u "$UNIT" --since '-15 min' --utc -o short-iso -p err \
  | awk '{print substr($1,1,16)}' | uniq -c

When the impact is unknown

Sometimes you genuinely cannot tell yet: the monitoring is part of the outage, the affected system has no external probe, or the symptom is a data-integrity question that needs a query to answer.

The rule is simple and frequently ignored: unknown impact is treated as high impact, with a clock on resolving the unknown.

1. Declare at the severity the worst plausible reading would justify, and say explicitly that it is provisional.
2. Name one person and one question: who is establishing the actual impact, and by when. Ten minutes is a normal answer.
3. Do not let the assessment become a background task. If the person assigned to measure impact gets pulled into the fix, the incident is now running blind and nobody has noticed.
4. Re-declare as soon as the measurement exists, up or down, out loud, in the channel.

The failure mode this avoids is the incident that runs for forty minutes at SEV3 because nobody could confirm it was worse, and is retrospectively found to have been a SEV1 the whole time. The cost of an over-declaration is an apology. The cost of an under-declaration is the response you did not assemble.

Impact is not a single reading

Declaring impact once, at the start, is the second most common error. Impact moves, and it usually moves in one direction:

• A queue backing up has small impact at minute two and total impact at minute forty, when the queue hits its bound and starts rejecting.
• A disk filling has zero impact until it has complete impact, and the transition is not gradual.
• A retry storm grows the impact of the original fault, so the incident gets worse while you work on it correctly.
• A partial mitigation reduces impact without resolving the incident, and the reduction can hide a still-growing fault underneath.

# The projection matters more than the current value. Two samples
# ten minutes apart give a slope you can act on.
df -h /var/lib/postgresql
sleep 600
df -h /var/lib/postgresql

Attach a re-assessment to the update cadence you already have: if the severity requires an update every 30 minutes, the impact figure in that update is a fresh reading, not the one from the first update copied forward.

Writing the assessment down

Put it in the ticket in a fixed shape so the next reader, and the post-incident review, can find it:

IMPACT ASSESSMENT - INC-4820
Taken at:      2026-08-11T06:12Z    By: e.brandi
Scope:         All customers in the EU region (approx 40% of traffic)
Depth:         Checkout unavailable; browse and search unaffected
Duration:      Started 05:58Z per edge error rate; 14 min at time of writing
Reversibility: No data loss. Payment attempts rejected before authorisation;
               no partial transactions observed in the payment log.
Measured by:   Edge 5xx rate (grafana/checkout-edge), confirmed with a
               30-sample curl loop from outside the platform.
Confidence:    High on scope and depth. Reversibility checked for the
               payment path only; the order-history write path has NOT
               been checked.
Severity:      SEV1 (declared 06:14Z)

The Confidence line is what makes this an assessment rather than a claim. Naming the path you did not check is how somebody else knows to go and check it.

Knowledge check