Skip to main content
RunBook Academy

← All break/fix scenarios in Observability

advancedgrafana-dashboard~30 min

Break/Fix: Dashboard Variable Causes Huge Query

Reported symptoms

  • ●Dashboard load is slow

Evidence

  • · Variables expand to many values
Diagnosis and resolutionclick to reveal

Root cause

A variable query returns unbounded results.

Remediation

1. Inspect variable definition. 2. Add limit. 3. Test load time.

Verification

Dashboard loads quickly.

Prevention

Cap variable values at reasonable limit

Scenario

You are operating a production observability platform. The following symptoms appear:

  • Dashboard load is slow

Available evidence:

  • Variables expand to many values

Your task

Determine the cause, recover, document, and validate.

Investigation

The investigation follows the discipline taught in Part XCVIII:

  1. Form hypothesis, find evidence, test, validate.
  2. Use the available evidence above to bound the search.
  3. Reach one of the likely root causes.

Recovery procedure

(Do not reveal until you have reasoned through the problem.)

  1. Identify the failing component.
  2. Apply the remediation pathway.
  3. Validate with the verification step.
  4. Document the incident.

Remediation

  1. Inspect variable definition. 2. Add limit. 3. Test load time.

Verification

Dashboard loads quickly.

Rollback

Revert

Prevention

Cap variable values at reasonable limit