Skip to main content
RunBook Academy

← All assessments

Final assessment · expert · ~180 min · pass ≥ 80%

Final Assessment: Production Observability Operations

Objectives

  • Pass a 90-minute theory assessment covering every part of the course
  • Diagnose a deliberately flawed observability environment
  • Identify operational risks in the configuration
  • Propose remediations for each finding
  • Rank the findings by blast radius
  • State the validation procedure for each remediation

0 questions · last verified 2026-08-13

Theory Assessment (90 minutes)

The theory assessment covers every part of the course. You must score 80% or higher to pass.

The questions test judgement rather than trivia. Examples:

  • Why is a high-cardinality metric label dangerous?
  • A service latency alert fires, but host CPU / memory / network metrics appear normal. Which telemetry signal is most useful?
  • A card transaction fails for 30% of users, but no alert fires. Where would you investigate?
  • What is the operational difference between monitoring and observability?
  • What is the wrong kind of alert to fire on a node?

Each question has a single correct answer (or, for multi-select, a specific list of correct answers).

Practical Assessment (90 minutes)

The practical assessment is a deliberately flawed production observability environment. You are asked to identify the flaws, propose remediations, and rank them by blast radius.

Findings to investigate

The estate has the following hidden issues. You must identify each:

  • High-cardinality metric (request_id label).
  • Bad PromQL (rate() with a window too large for a counter reset).
  • Broken alert routing (every alert fires but nothing notifies).
  • Alert with no for: clause (flapping).
  • Alert with no severity / PagerDuty mapping.
  • Alert with broad labels catching unrelated firing alerts.
  • Grafana anonymous admin access.
  • Loki retention set to 24h when SLO is 7 days.
  • Loki label using request_id (high cardinality).
  • Trace context broken in client (no traceparent header).
  • Trace sampling at 100% in production.
  • Prometheus TSDB 95% full.
  • No backup of alert rules / dashboards.
  • No monitoring of the monitoring stack.
  • TLS expiry alert only fires at 1 day (no early warning).
  • Collector forwarding without batching / backpressure.
  • All dashboards targeted by instance=~".*" (regex DoS).
  • Alert without runbook link.
  • Sensitive value (PII, payment) detected in a log line.
  • No meta-monitoring.
  • Loki storage compaction disabled.

Deliverables

For each finding:

  1. Evidence.
  2. Root cause.
  3. Impact.
  4. Remediation.
  5. Validation procedure.
  6. Prevention strategy.

Rank all findings by blast radius. The highest-blast-radius item is the most important to fix first.

Pass Criteria

  • Theory: >= 80%.
  • Practical: all 21 findings correctly identified.

A pass on both halves is required to complete the course.