Subsystem by subsystem — kernel, FRR, firewall, interface, application, isolate layer

A VyOS 1.5 LTS router is composed of five subsystems: the Linux kernel (network stack, routing table, FIB), FRR (routing protocols), the firewall (nftables), the interfaces (NICs, VLANs, tunnels), and the applications (BGP, OSPF, IPsec, etc.). A failure in any subsystem can manifest as the same symptom (e.g., “external connectivity fails”). The operator’s discipline is to isolate the failing subsystem before investigating within it.

This lesson is the fourth in Part LII: the canonical diagnostic for each subsystem, the discipline of subsystem isolation, and the production workflow that converts a complex symptom into a single-subsystem root cause.

The five subsystems

flowchart TB
  subgraph APP["Application layer"]
    BGPD["bgpd"]
    OSPFD["ospfd"]
    SWAN["strongSwan"]
  end
  subgraph FRR["FRR layer"]
    Z["zebra<br/>(RIB)"]
  end
  subgraph FW["Firewall layer"]
    NF["nftables"]
    CT["conntrack"]
  end
  subgraph KERNEL["Kernel layer"]
    FIB["FIB"]
    NET["netfilter hooks"]
    ROUTE["routing table"]
  end
  subgraph IF["Interface layer"]
    ETH["eth0 / eth0.100"]
    WG["wg0"]
    VTI["vti0"]
  end
  APP --> FRR
  FRR --> KERNEL
  FW --> KERNEL
  KERNEL --> IF

Each subsystem has a distinct diagnostic. The operator runs the diagnostic for each subsystem to identify which is failing.

Subsystem 1: Interface layer

The interface layer is the NICs, VLANs, tunnels, and bonds. The diagnostic:

show interfaces
# Per-interface state, MTU, counters, errors
ip link show
# Kernel's view of the interface
ip -s link show <intf>
# Detailed counters
ethtool <intf>
# Speed, duplex, link state
ethtool -S <intf>
# Driver-specific counters
ethtool -k <intf>
# Offload features

What to look for:

• Link state. Is the interface up or down? A down interface means no traffic flows.
• MTU. Is the MTU configured correctly? An MTU mismatch causes large packets to fail.
• Errors. rx_errors, tx_errors, rx_crc_errors, rx_missed_errors. Errors indicate a physical-layer problem (cable, SFP, switch port).
• Drops. rx_dropped, tx_dropped. Drops indicate buffer exhaustion or congestion.
• Speed and duplex. ethtool <intf> shows the negotiated speed and duplex. A 100 Mbps link instead of 1 Gbps indicates a physical-layer problem.

A failed interface means the issue is at the interface layer; the operator investigates the cable, SFP, switch port, or NIC.

Subsystem 2: Kernel layer

The kernel layer is the Linux network stack: routing table, FIB, netfilter hooks, neighbour table. The diagnostic:

show ip route
# Active routing table (from FRR zebra)
ip route show
# Kernel's routing table (may differ from FRR's view)
ip route show table all
# All routing tables (main, local, default)
ip rule show
# Routing rules (policy routing)
ip neighbor show
# ARP/ND table
cat /proc/net/softnet_stat
# Per-CPU packet-processing stats
ss -s
# Socket statistics

What to look for:

• Routing table. Does the route to the destination exist? Is the next-hop reachable? Is the metric correct?
• FIB. Is the route installed in the kernel’s FIB (ip route show)? FRR’s RIB and the kernel’s FIB may diverge if there is a synchronization issue.
• Routing rules. Are the routing rules correct? Policy routing can send traffic to a different table.
• ARP/ND. Is the next-hop resolved? An incomplete ARP entry means the next-hop is unreachable.
• Socket statistics. Are sockets accumulating? High socket counts may indicate a leak.

A kernel-layer failure means the issue is in routing (configuration, FRR-kernel sync, ARP); the operator investigates the routing configuration and the kernel’s view of the routes.

Subsystem 3: FRR layer

The FRR layer is the routing protocols: zebra, bgpd, ospfd, ripd, isisd, etc. The diagnostic:

show ip bgp summary
# BGP peer state
show ip bgp <prefix>
# BGP route details
show ip ospf neighbor
# OSPF neighbour state
show ip ospf database
# OSPF LSA database
show ip ospf interface
# OSPF interface state
show process cpu
# FRR process CPU usage
show thread cpu
# FRR thread CPU usage
vtysh -c 'show running-config'
# Full FRR configuration
journalctl -u frr --since "1 hour ago"
# FRR logs

What to look for:

• Peer state. Is the BGP session in Established or in another state? An Active state means the session is not up.
• Neighbour state. Is the OSPF neighbour in Full or in another state? Init or 2-Way indicates a problem.
• Route table. Does FRR have the route? show ip bgp <prefix> or show ip route shows the FRR view.
• Logs. What do the FRR logs say? The logs identify the specific reason for a flap or failure.
• CPU usage. Is FRR consuming CPU? A high FRR CPU indicates route churn or other routing-protocol issues.

A FRR-layer failure means the issue is in the routing protocol configuration or peer state; the operator investigates the configuration, peer reachability, and FRR logs.

Subsystem 4: Firewall layer

The firewall layer is nftables (the VyOS 1.5 LTS firewall) and conntrack. The diagnostic:

show firewall
# Firewall ruleset summary
show firewall name <name>
# Specific chain
nft list ruleset
# Full nftables ruleset
conntrack -L
# Connection tracking table
conntrack -S
# Connection tracking statistics

What to look for:

• Rule order. Are the rules in the correct order? A default-action drop before an accept rule causes the accept rule to be unreachable.
• Counter mismatches. Do the counters show the expected traffic? A rule with 0 packets means the rule is not matching.
• Conntrack table. Is the conntrack table filling up? A full table causes new connections to be dropped.
• Conntrack entries. Is the connection being tracked? A connection without a conntrack entry may be a stateful-firewall issue.

A firewall-layer failure means the issue is in the firewall configuration or state table; the operator investigates the rules, counters, and conntrack.

Subsystem 5: Application layer

The application layer is the daemon-specific state: bgpd, ospfd, strongSwan, wg-quick, etc. The diagnostic:

# For bgpd
show ip bgp neighbor <peer>
show ip bgp regexp <pattern>
journalctl -u frr --since "1 hour ago"

# For strongSwan
swanctl --list-sas
swanctl --list-pols
journalctl -u strongswan --since "1 hour ago"

# For WireGuard
wg show
journalctl -u wg-quick@<intf> --since "1 hour ago"

What to look for:

• Session state. Is the application session in the expected state? An IPsec SA in INSTALLED is correct; in REKEYING may indicate a problem.
• Logs. What do the application logs say? The logs identify the specific reason for a failure.
• Counters. What are the application-specific counters? BGP message counts, IPsec byte counts, etc.

An application-layer failure means the issue is in the application-specific state; the operator investigates the application logs and configuration.

The isolation workflow

sequenceDiagram
  participant Operator
  participant IF as Interface
  participant Kernel
  participant FRR
  participant FW as Firewall
  participant App as Application

  Operator->>IF: Step 1: show interfaces
  IF-->>Operator: Link up, no errors
  Operator->>Kernel: Step 2: ip route show
  Kernel-->>Operator: Route present
  Operator->>FRR: Step 3: show ip bgp summary
  FRR-->>Operator: BGP session Active
  Operator->>FW: Step 4: show firewall
  FW-->>Operator: Rules correct, counters matching
  Operator->>App: Step 5: app-specific check
  App-->>Operator: App state correct

The operator walks each subsystem in order. The first subsystem that shows unexpected state is the failure point.

The discipline: do not skip subsystems. An operator who sees the FRR state is wrong and immediately investigates FRR without checking the interface layer may miss an interface-down issue.

Cross-subsystem symptoms

Some symptoms span multiple subsystems:

• BGP session down + interface up. The interface is up (ping succeeds) but the BGP session is in Active. The issue is at the FRR layer (configuration, peer) or application layer (bgpd state).
• Routing table correct + traffic fails. The routing table has the correct route, but traffic does not flow. The issue is at the firewall layer (firewall blocking) or interface layer (MTU).
• FRR state correct + kernel FIB empty. FRR has the route but the kernel’s FIB does not. The issue is at the kernel layer (FRR-kernel sync).

The operator must check each subsystem to find the cross-subsystem interaction.

Production failure modes

The subsystem-by-subsystem failure modes the operator encounters:

• Skipping subsystems. The operator sees an FRR issue and investigates only FRR; the actual issue is in the interface layer. Fix: check every subsystem in order.
• Misreading subsystem state. The operator sees show ip bgp summary showing Active and concludes the issue is at the FRR layer; the actual issue is at the firewall layer (TCP 179 blocked). Fix: check the firewall state.
• Ignoring counters. The operator sees show ip bgp summary showing Established and concludes the session is up; the counters show 0 messages received. Fix: check the counters.
• Skipping the kernel layer. The operator sees FRR state correct and concludes the issue is at the application layer; the kernel’s FIB is empty. Fix: check the kernel’s view of the routes.
• Skipping the interface layer. The operator sees BGP state correct and concludes the issue is elsewhere; the interface is down. Fix: always check the interface layer first.

Rollback

Subsystem-by-subsystem changes are about analysis, not rollback. The discipline:

• Capture the diagnostic output for each subsystem.
• Identify the failing subsystem from the diagnostic.
• Apply the fix at the failing subsystem.
• Validate by re-running the diagnostic.

Production discipline

Cross-course references

• Part LII-01 (LII-VyOS-Troubleshoot / define-and-scope) covers the incident-definition that precedes subsystem isolation.
• Part LII-02 (LII-VyOS-Troubleshoot / evidence-first) covers the evidence collection that precedes subsystem isolation.
• Part LII-03 (LII-VyOS-Troubleshoot / hypothesis-driven) covers the hypothesis formation that precedes subsystem isolation.
• Part III (III-VyOS-Architecture) covers the VyOS architecture (kernel, FRR, firewall, interface, application).
• Part XXXVII (XXXVII-VyOS-Firewall) covers the firewall primitives.
• Part XXIV (XXIV-VyOS-BGP) and Part XIX (XIX-VyOS-OSPF) cover the routing protocol primitives.

Quiz