Define and scope — ticket triage, scope boundaries, who is affected

Every troubleshooting engagement starts with a question: what is broken, and for whom? The operator who jumps to a hypothesis before answering these two questions will spend hours chasing the wrong root cause. The operator who takes five minutes to define the problem and scope it will resolve the issue in the right subsystem and avoid wasted effort.

This lesson is the first in Part LII: the discipline of defining the problem and scoping it before any hypothesis or fix. The lesson establishes the vocabulary (severity, urgency, priority, scope, impact) and the workflow that all subsequent troubleshooting follows.

The two questions

flowchart LR
  T["Ticket arrives<br/>('the network is broken')"]
  T --> Q1["Q1: What is broken?<br/>(symptom, evidence)"]
  Q1 --> Q2["Q2: For whom is it broken?<br/>(scope, impact)"]
  Q2 --> H["Form hypotheses"]
  H --> Test["Test"]
  Test --> Fix["Fix or rollback"]

Q1: What is broken? The operator describes the symptom in observable terms. “Users cannot reach 198.51.100.1” is a symptom. “The router’s BGP session to ISP-A is down” is a symptom. “The network is slow” is not a symptom — it is a complaint. The operator’s first job is to convert complaints into symptoms.

Q2: For whom is it broken? The operator identifies the scope. “One user cannot reach 198.51.100.1” is a single-user scope. “All users in the engineering VLAN cannot reach 198.51.100.1” is a site scope. “All users across all VLANs cannot reach any external destination” is a network-wide scope. The scope determines the urgency and the priority of the response.

The two questions are independent. A network-wide outage may have a low-severity symptom (degraded performance rather than full loss). A single-user outage may have a high-severity symptom (the user is the CEO). The operator must answer both questions before responding.

Severity, urgency, priority

Three concepts that are often confused:

Severity. The technical impact of the problem. A BGP session to a transit provider is down: severity is high (the network cannot reach external destinations). A BGP session to a backup peer is down: severity is medium (the network has redundancy). A BGP session to a partner network is down: severity is low (the partner is non-critical).

Urgency. The time pressure to resolve. A production database is unreachable: urgency is high (every minute of downtime costs money). A non-production testbed is down: urgency is low (the testbed can wait).

Priority. The combination of severity and urgency. A high-severity, high-urgency issue is priority 1 (P1). A low-severity, low-urgency issue is priority 4 (P4). The priority determines who responds and how fast.

flowchart TD
  S["Severity (technical impact)"] --> P["Priority"]
  U["Urgency (time pressure)"] --> P
  P --> R["Response"]
  P --> P1["P1: < 15 min"]
  P --> P2["P2: < 1 hour"]
  P --> P3["P3: < 4 hours"]
  P --> P4["P4: < 24 hours"]

The discipline: a low-severity, high-urgency issue is still high priority (e.g., a non-critical application is down but the user is the CEO). A high-severity, low-urgency issue is also high priority if the impact will grow (e.g., a BGP session is flapping; severity is medium now, but if it goes down completely, the network is offline).

Scope boundaries

The operator identifies the scope with three questions:

1. Who is affected? One user, one site, one VLAN, all users, all sites, specific partner, specific application.
2. What is affected? Connectivity, performance, specific application, specific destination.
3. When did it start? Right now (regression), recently (configuration change), long-standing (design issue).

flowchart TD
  S["Scope"]
  S --> W["Who"]
  S --> W2["What"]
  S --> W3["When"]
  W --> W1["Single user / site / network"]
  W2 --> W2A["Connectivity / performance / application"]
  W3 --> W3A["Right now / recently / long-standing"]

The three answers determine the response:

• Single user + connectivity + right now. Likely a host-side issue (DNS, IP address, default gateway). The operator investigates the host, not the router.
• Site + connectivity + right now. Likely a site-side issue (link down, switch issue). The operator investigates the site uplink.
• Network-wide + connectivity + right now. Likely a core issue (router, BGP, default route). The operator investigates the core.
• Single user + performance + recently. Likely a recent change on the user’s host or path. The operator investigates recent changes.
• Site + performance + long-standing. Likely a design issue (under-provisioned link, congestion). The operator investigates capacity.

The discipline: the scope answers determine which subsystem to investigate. The operator who investigates a host issue at the router level, or a router issue at the host level, is wasting time.

Impact assessment

The operator assesses the impact:

• Direct impact. The symptom: users cannot reach a specific destination, or a specific application is down.
• Indirect impact. What depends on the directly-impacted service: a database is unreachable, so all applications that use the database are down.
• Escalation risk. What will happen if the issue is not resolved soon: a non-critical issue becomes a critical issue, a site outage becomes a network-wide outage.

flowchart LR
  D["Direct impact<br/>(symptom)"] --> I["Indirect impact<br/>(what depends on it)"]
  I --> E["Escalation risk<br/>(what happens if not resolved)"]

A concrete example: a BGP session to a transit provider is flapping (direct impact: loss of external connectivity). The indirect impact: all external destinations are intermittently unreachable. The escalation risk: if the flapping continues, downstream BGP sessions will reset, and the network will lose all external connectivity.

The discipline: the operator who treats the direct impact without assessing the indirect impact and the escalation risk may resolve the immediate symptom but miss the larger issue. The operator who escalates without assessing the direct impact may over-respond to a non-issue.

The workflow

The complete incident-response workflow:

sequenceDiagram
  participant Operator
  participant System as VyOS router

  Operator->>Operator: 1. Define the problem<br/>(symptom, evidence)
  Operator->>Operator: 2. Scope the impact<br/>(who, what, when)
  Operator->>Operator: 3. Form hypotheses<br/>(subsystems)
  Operator->>System: 4. Collect evidence<br/>(logs, counters, state)
  Operator->>Operator: 5. Test hypotheses<br/>(one at a time)
  Operator->>Operator: 6. Identify root cause
  Operator->>System: 7. Apply fix
  Operator->>System: 8. Validate fix
  Operator->>Operator: 9. Document postmortem

The first two steps (define and scope) are the subject of this lesson. Steps 3-6 are the subject of subsequent lessons (Part LII-03 hypothesis-driven). Steps 7-9 are the subject of subsequent lessons (Part VI commit and rollback).

Production failure modes

The define-and-scope failure modes the operator encounters:

• Scope confusion. The operator believes the issue is network-wide when it is site-wide; the operator spends hours investigating the core when the issue is on a single uplink. Fix: scope the impact before investigating.
• Symptom-as-complaint. The operator chases a complaint (“the network is slow”) without converting it to a symptom. Fix: convert complaints to symptoms (latency, packet loss, throughput).
• Severity-urgency confusion. The operator treats a high-severity, low-urgency issue as low priority, or a low-severity, high-urgency issue as low priority. Fix: assess severity and urgency independently; combine into priority.
• Impact over-assessment. The operator escalates a low-impact issue to P1 because the user is “important”. Fix: assess impact by technical effect, not by user identity.
• Impact under-assessment. The operator treats a high-impact issue as low priority because the immediate symptom is small. Fix: assess indirect impact and escalation risk.
• Premature hypothesis. The operator forms a hypothesis (e.g., “the BGP session is down”) before defining the problem. Fix: define and scope first; form hypotheses after.

Rollback

Define-and-scope changes are about analysis, not rollback. The discipline:

• Document the symptom and scope in the ticket.
• Update the ticket as the scope expands or contracts.
• Communicate the scope to stakeholders (users, management, vendors).

Production discipline

Cross-course references

• Part LII-02 (LII-VyOS-Troubleshoot / evidence-first) covers the evidence collection that follows define-and-scope.
• Part LII-03 (LII-VyOS-Troubleshoot / hypothesis-driven) covers the hypothesis formation that follows define-and-scope.
• Part LII-04 (LII-VyOS-Troubleshoot / subsystem-by-subsystem) covers the subsystem isolation that follows define-and-scope.
• Part LII-05 (LII-VyOS-Troubleshoot / return-path) covers the return-path verification that complements define-and-scope.
• Part LII-06 (LII-VyOS-Troubleshoot / troubleshooting anti-patterns) covers the anti-patterns of define-and-scope (skip, jump to hypothesis).
• The Observability course covers the monitoring and alerting side (incident detection, alert triage).

Quiz