Firewall troubleshooting — log, conntrack, packet capture, the diagnostic method

The firewall is the most security-sensitive subsystem on a production router. When it does not admit traffic it should, the operator must diagnose quickly. When it admits traffic it should not, the operator must diagnose immediately.

The diagnostic method walks the chain from ingress to egress. The operator uses three tools: the firewall log (find which rule fired), the conntrack table (verify the flow state), and packet capture (prove whether the packet reaches the router).

This lesson covers the diagnostic method, the three tools, the production failure modes where the firewall appears to be working but is silently bypassed, and the anti-patterns the operator must avoid during diagnosis.

The diagnostic method

The firewall diagnostic method is a five-step walk:

1. Reproduce — confirm the failure with a known traffic pattern. The operator must be able to reproduce the problem on demand; intermittent failures cannot be diagnosed.
2. Identify the chain — which chain should see this packet? On VyOS 1.5 that is two questions, not one. First the base hook: input filter for traffic addressed to the router, forward filter for traffic passing through it, output filter for traffic the router originates. Then, within that hook, whether a rule with action jump sends the packet to a named rule-set — and whether that rule’s interface and address matches actually cover this packet.
3. Verify the chain is reached — does the chain see the packet? Use packet capture to confirm.
4. Find the rule that fired — use the firewall log and rule hit counters to identify the rule.
5. Verify the conntrack state — if the packet should be ESTABLISHED but is judged as NEW, the conntrack table is the issue.

flowchart TB
  S1["Step 1: Reproduce<br/>known-good traffic<br/>known-bad result"]
  S2["Step 2: Identify the chain<br/>which base hook?<br/>which jump, if any?"]
  S3["Step 3: Verify the chain is reached<br/>tcpdump on ingress<br/>rule hit counter advancing?"]
  S4["Step 4: Find the rule that fired<br/>firewall log<br/>hit counter on each rule"]
  S5["Step 5: Verify conntrack state<br/>conntrack -L<br/>conntrack -E"]
  FIX["Diagnosis complete<br/>fix and re-test"]

  S1 --> S2
  S2 --> S3
  S3 -->|"no"| RX["Diagnosis: packet not<br/>reaching the router<br/>check L2/L3 path"]
  S3 -->|"yes"| S4
  S4 -->|"rule identified"| S5
  S4 -->|"rule not identified"| CAP["Diagnosis: rule not logging<br/>enable log, retest"]
  S5 -->|"state correct"| FIX
  S5 -->|"state wrong"| CT["Diagnosis: conntrack issue<br/>check table size,<br/>check timeouts"]

The five steps are sequential. Skipping a step leads to a misdiagnosis. The production discipline: every firewall diagnosis follows all five steps.

Tool 1: the firewall log

The firewall log is the operator’s primary diagnostic tool. On VyOS 1.5 a rule logs when it carries the valueless log node, and the level it logs at is log-options level:

set firewall ipv4 name WAN-IN rule 20 log
set firewall ipv4 name WAN-IN rule 20 log-options level 'info'
set firewall ipv4 name WAN-IN default-log

default-log logs what falls off the end of the rule-set into default-action, which is the case a per-rule log cannot cover because no rule fired.

Reading the log:

# The whole firewall log
show log firewall

# Narrowed to one rule-set, which is what you almost always want
show log firewall ipv4 name WAN-IN

# Narrowed to a single rule
show log firewall ipv4 name WAN-IN rule 20

# Narrowed to a base hook
show log firewall ipv4 forward filter

# The last hundred lines
show log firewall | last 100

# Follow the log live while you reproduce the failure
monitor log

A log line carries the ingress and egress interfaces, the addresses, the protocol and ports, the TCP flags, and a bracketed prefix that encodes which rule fired:

$ show log firewall ipv4 name WAN-IN

Aug 19 14:23:45 router1 kernel: [ruleset-rule-action] IN=eth0 OUT=eth1 SRC=198.51.100.50 DST=203.0.113.10 PROTO=TCP SPT=52012 DPT=22 WINDOW=65535 RES=0x00 SYN URGP=0

The fields after the prefix are netfilter’s, and they are stable: IN= and OUT= are the ingress and egress interfaces, SRC/DST the addresses, PROTO/SPT/DPT the protocol and ports, and the bare flag words (SYN here) the TCP flags.

The bracketed prefix is the part to read on your own box rather than from a document. VyOS composes it from the address family, the rule-set, the rule number and the action, and the exact composition has changed between releases — so the reliable move is to make a rule you control fire once, find its line, and learn the shape from that. Once you have it, the prefix is what turns “something dropped this” into “rule 20 of WAN-IN dropped this”, which is the whole value of the log.

SYN with no ACK on a dropped line is worth a second glance: it means the connection never got started, so the problem is inbound policy. A dropped line with ACK set is usually the return half of a flow whose state was lost, which points at conntrack rather than at the rule-set.

Tool 2: rule hit counters

The firewall rule hit counters show how many times each rule has fired. A counter that does not advance means the rule is not firing; a counter that advances on the wrong rule means the rule ordering is wrong.

# All rules in a named rule-set, with hit counters
show firewall ipv4 name WAN-IN

# One rule
show firewall ipv4 name WAN-IN rule 10

# The base hooks - including whether the jump rule is firing
show firewall ipv4 forward filter
show firewall ipv4 input filter

# Aggregated counters
show firewall statistics

A typical output:

$ show firewall ipv4 name WAN-IN

Ruleset Information

ipv4 Firewall "name WAN-IN"

Rule     Action    Protocol    Packets    Bytes    Conditions
-------  --------  ----------  ---------  -------  ------------------------------
10       accept    all         1845234    2.3G     state established,related
20       drop      all         12         1.2K     state invalid
30       accept    tcp         432        28K      saddr 198.51.100.0/24 dport 22
40       accept    tcp         234982     28M      daddr 203.0.113.10 dport 443
default  drop                  847213     95M

The hit counters tell the operator:

• Rule 10 (established/related) — 1.8M packets. This is return traffic; the counter advancing means conntrack is working.
• Rule 20 (INVALID) — 12 packets. A small number; consistent with normal scanner traffic.
• Rule 30 (SSH) — 432 packets. A few SSH connections per day; consistent with operator access.
• Rule 40 (HTTPS) — 235K packets. The web server’s traffic.
• default — 847K packets. This is default-action drop rather than a numbered rule, which is why default-log and not a per-rule log is what makes it visible.

The discipline: the default-action counter is the operator’s window into the firewall’s effectiveness, and it is read for its behaviour, not its value. A large number on a WAN-facing rule-set is normal — it is the Internet’s background scanning. What matters is the shape of the change:

• A default counter that stops advancing on a WAN-facing rule-set means something above it is now matching everything. That is the silent bypass in this lesson, and it looks like good news on a graph.
• A default counter that jumps by orders of magnitude means either a scan or a legitimate service that just stopped being permitted. The log tells you which.
• A rule counter that never moves at all is dead code: a permit for a service that was decommissioned, or a rule shadowed by one above it. Both are worth knowing about and neither is visible in the configuration.

Tool 3: conntrack

The conntrack table is the source of truth for the firewall’s stateful behaviour. The operator uses conntrack -L to list the current entries and conntrack -E to see live events.

VyOS exposes the table from operational mode, which is the first place to look because it needs no sudo and no recollection of conntrack flags:

show conntrack table ipv4
show conntrack statistics

show conntrack statistics is the one that answers “is the table the problem” — it reports the current count, and the count against the maximum is the number that matters when new flows are failing and established ones are not.

For anything more selective, the conntrack tool underneath takes filters:

# List all conntrack entries
sudo conntrack -L

# List entries from a specific source
sudo conntrack -L -s 198.51.100.50

# List entries to a specific destination
sudo conntrack -L -d 203.0.113.10

# Live event stream
sudo conntrack -E

# Show the table size and current count
cat /proc/sys/net/netfilter/nf_conntrack_max
cat /proc/sys/net/netfilter/nf_conntrack_count

# Delete a specific conntrack entry (force re-evaluation)
sudo conntrack -D -s 198.51.100.50 -d 203.0.113.10

# Delete all entries for a specific source
sudo conntrack -D -s 198.51.100.50

A typical conntrack -L output:

tcp 6 300 ESTABLISHED src=198.51.100.5 dst=203.0.113.10
  sport=52012 dport=22 packets=12 bytes=1844 [ASSURED]
  src=203.0.113.10 dst=198.51.100.5 sport=22 dport=52012
  packets=8 bytes=4096

The fields:

• tcp 6 300 — protocol (tcp), protocol number (6), and timeout (300 seconds remaining).
• ESTABLISHED — the state.
• src=... dst=... sport=... dport=... — the four-tuple (source address, destination address, source port, destination port).
• packets=12 bytes=1844 — the packet count and byte count for this direction.
• [ASSURED] — the flow has been confirmed bidirectional.

The discipline: a packet that should be ESTABLISHED but is not in the conntrack table is a conntrack issue rather than a rule-set issue, and the two produce very different symptoms. A rule-set problem fails the same way every time. A conntrack problem fails for new flows while existing ones keep working, which is the signature to learn.

The operator checks the count against the maximum (show conntrack statistics), the timeouts, and the kernel’s own complaint: dmesg | grep nf_conntrack prints a table full, dropping packet line when the table is the cause. That message is conclusive; everything else is inference.

The knobs, when the table genuinely needs to be larger:

set system conntrack table-size 524288
set system conntrack timeout tcp established 3600

Raise the table size only after finding out what filled it. A table full of half-open connections from one source is a scan, and the answer is a firewall rule, not more memory.

Tool 4: packet capture

Packet capture with tcpdump proves whether the packet reaches the router. The operator uses tcpdump on the ingress interface to confirm the packet arrives, and on the egress interface to confirm the packet leaves.

# Capture on the ingress interface
sudo tcpdump -ni eth0 'host 198.51.100.50 and port 22'

# Capture on the egress interface
sudo tcpdump -ni eth1 'host 203.0.113.10 and port 22'

# Capture with verbose output (shows flags)
sudo tcpdump -ni eth0 -vv 'host 198.51.100.50 and port 22'

# Capture and write to a file for analysis
sudo tcpdump -ni eth0 -w /tmp/capture.pcap 'host 198.51.100.50'

# Read the capture
sudo tcpdump -nr /tmp/capture.pcap

# Capture with nftables tracing
sudo nft monitor

A typical tcpdump output:

14:23:45.123456 IP 198.51.100.50.52012 > 203.0.113.10.22: Flags [S], seq 12345, win 65535, length 0
14:23:45.124567 IP 203.0.113.10.22 > 198.51.100.50.52012: Flags [S.], seq 98765, ack 12346, win 65535, length 0

The first line shows the SYN arriving on the ingress interface. The second line shows the SYN/ACK leaving the router (a response). If the second line is missing, the firewall dropped the SYN.

The discipline: tcpdump is the operator’s proof that the packet reached the firewall. Without tcpdump, the operator is guessing whether the packet arrived at all.

The silent bypass failure

The silent bypass is the most dangerous firewall failure: the firewall appears to be working (the counters are advancing, the log is being written) but is silently admitting traffic it should not.

The four common silent bypass failure modes:

1. The rule-set is never reached. A set firewall ipv4 name WAN-IN ... rule-set is inert on its own. Nothing in it is evaluated until a base hook contains a rule with action jump and a jump-target naming it. A forty-rule WAN-IN that no hook jumps to reads perfectly in show configuration and filters nothing, and its counters sit at zero — which is the tell, if anyone looks.
2. The jump is in the wrong hook, or matches the wrong interface. A jump written in input filter catches traffic addressed to the router and misses everything passing through it; a jump whose inbound-interface name names the wrong interface catches nothing at all. Both produce a rule-set that is reached for some traffic, which is more confusing than one that is never reached. The same trap exists in the zone model: set firewall zone is direction-specific, so a policy written for WAN from LAN says nothing about LAN from WAN.
3. Shadowed rule. A permissive rule above the intended deny matches first, so the deny never fires. Its counter stays at zero while the rule above it climbs.
4. Missing state clause. A rule that accepts without a state qualifier accepts in any state, including packets belonging to no tracked flow. On 1.5 the value stands alone — state new, state established, state related, state invalid — and the 1.3 shape with an enable leaf under state does not commit.
5. NAT ordering. Destination NAT runs before the forward filter, so a forward rule sees the translated destination; source NAT runs after, so the same rule sees the original source. A rule written against the address the operator has in their head, rather than the address the chain sees at that point, matches nothing and is invisible in review.

The discipline: every silent bypass is detected by a test. The operator attempts to send a packet that should be denied and confirms it is denied. The operator attempts to send a packet that should be admitted and confirms it is admitted.

# From an arbitrary Internet source, attempt SSH - must be denied
ssh -o ConnectTimeout=5 netops@203.0.113.10

# From an arbitrary Internet source, attempt HTTPS - must be admitted
curl -sS -o /dev/null -w '%{http_code}\n' https://203.0.113.10/

# From the jump host, attempt SSH - must be admitted
ssh -o ConnectTimeout=5 netops@203.0.113.10

A drop and a reject look different from the client, and the difference is worth knowing before you interpret the result. action drop produces a timeout — the client waits out ConnectTimeout and gives up. action reject produces an immediate refusal. A test that returns instantly when you expected a timeout has not proven the traffic was denied by the rule you think; it may have been refused by the host behind the firewall, which means the packet got through.

The discipline: the test matrix (denied, admitted, denied) is the operator’s proof that the firewall is enforcing the intended policy.

Operational commands

The firewall troubleshoot lesson uses the following operational commands:

# Firewall log, narrowed
show log firewall
show log firewall ipv4 name WAN-IN
show log firewall ipv4 name WAN-IN rule 20
monitor log

# Rule-sets, base hooks, and their counters
show firewall ipv4 name WAN-IN
show firewall ipv4 name WAN-IN rule 10
show firewall ipv4 forward filter
show firewall ipv4 input filter
show firewall statistics

# Conntrack, from the CLI and from the tool underneath
show conntrack table ipv4
show conntrack statistics
sudo conntrack -L -s 198.51.100.50
sudo conntrack -E
sudo conntrack -D -s 198.51.100.50

# Packet capture
sudo tcpdump -ni eth0 'host 198.51.100.50 and port 22'
sudo tcpdump -ni eth0 -w /tmp/capture.pcap 'host 198.51.100.50'

# The generated ruleset - read the real table and chain names here
sudo nft list ruleset

Rollback

The firewall diagnostic changes (e.g., enabling tracing on a chain) are rolled back the same way as any VyOS configuration change:

A diagnostic change made through the CLI is reverted like any other, and narrowly:

delete firewall ipv4 name WAN-IN rule 20 log
commit
save

A rule added by hand with nft is a different matter, because it never existed in the configuration. It is removed with nft — sudo nft list ruleset -a prints the handles needed to delete one — and it is also removed, silently and whether you meant it or not, by the next commit. Neither route is a reason to leave one running.

Avoid rollback N here. It applies the stored revision and then restarts the router, which on a firewall diagnosis means dropping every session on the box to undo a log node. Where a broader revert is genuinely wanted, load the pre-change snapshot, compare, and commit-confirm.

The discipline: trace rules and log nodes are temporary. Remove them when the diagnosis ends — a forgotten log on a high-rate rule fills the disk, and a forgotten hand-added nft rule is a firewall rule with no representation in the configuration, invisible to every review that follows.

Production discipline

Cross-course references

• Part XXXVII-01 (XXXVII-VyOS-Firewall / stateful vs stateless) covers the conntrack layer.
• Part XXXVII-02 (XXXVII-VyOS-Firewall / zones and chains) covers the base hooks, the action jump model, and the firewall zone tree that replaced zone-policy in 1.4.
• Part XXXVII-05 (XXXVII-VyOS-Firewall / default deny) covers the canonical WAN-IN chain.
• Part LII (LII-VyOS-Troubleshoot) covers the broader troubleshooting methodology.
• Part XLVIII-03 (XLVIII-VyOS-Logging / firewall logs) covers the log forwarding and analysis.

Quiz