Skip to main content
RunBook Academy

← All labs in OPNsense

Lab · advanced · ~75 min

Lab: Capture packets on OPNsense and troubleshoot a broken connection

B · Nested virtualisationC · Simulation

Objectives

  • Capture on the right OPNsense interface (LAN, WAN, VLAN, loopback) for a given question
  • Read the captured pcap on OPNsense without copying it to a workstation
  • Diagnose a deliberately broken NAT reflection rule using a packet capture
  • Diagnose a deliberately broken firewall rule using a packet capture
  • Produce a short report tying symptom, layer, evidence, and likely cause together

Prerequisites

This lab uses OPNsense’s packet capture to diagnose three deliberate breakages: a NAT reflection rule that fails on a specific source address, a firewall rule that drops legitimate return traffic, and a host that is on the wrong VLAN. By the end you will be able to decide which OPNsense interface to capture on, run tcpdump there without locking up the GUI, and read the pcap well enough to distinguish the failure signatures.

OPNsense’s GUI has a packet capture tool under Diagnostics → Packet Capture, but the shell version is more flexible and is what production operators fall back on. The lab uses the shell.

Objective

By the end of this lab, you can:

  • Pick the correct OPNsense interface to capture on for a given troubleshooting question.
  • Run tcpdump with the right filters, host and port keywords, and stop conditions.
  • Read a pcap file on OPNsense without copying it off.
  • Distinguish the failure signatures of NAT reflection, a firewall drop, and a wrong-VLAN host.
  • Produce a one-page report tying symptom, layer, evidence, and likely cause together.

Requirements

You need:

  • An OPNsense instance with at least LAN, WAN, and one VLAN interface.
  • A host on each of the LAN and the VLAN that can produce traffic.
  • A reachable external service for the NAT reflection scenario — a public IP you control, or OPNsense’s own WAN IP for the hairpin scenario.
  • Shell access on OPNsense.
  • Enough free space under /root or /tmp for pcap files (~10 MB per scenario).

Tasks

Task 1: Identify the interface names

OPNsense’s interface names come from the FreeBSD driver that backs the NIC. The trap is that the GUI’s “LAN” can map to em0, igb0, vtnet0, or bge0 depending on hardware. Always confirm with ifconfig before capturing:

ifconfig | grep -E '^[a-z]' | awk '{print $1}'
ifconfig em0 | grep 'inet '

Write down the mapping. This lab assumes:

LAN  -> em0
WAN  -> em1
VLAN20 -> em0.20 (a VLAN subinterface on the LAN parent)
lo0  -> loopback

Task 2: Capture on LAN — prove the host is generating traffic

This is the diagnostic every troubleshooting session should start with: is the host on the LAN actually generating packets? The capture goes on OPNsense’s LAN interface; you do not need shell access to the host.

From OPNsense:

ssh root@10.10.10.1

# Capture for 30 seconds, stop with Ctrl-C
timeout 30 tcpdump -i em0 -nn -s 0 -w /root/cap_lan.pcap \
    'host 10.10.10.100'

From the LAN host:

curl -m 5 http://example.com/ 2>&1 | head -1

The capture must show the LAN host’s TCP/80 SYN going to 93.184.216.34 (or whatever example.com resolves to) on em0. If it shows nothing, the host is not on the LAN at all, or its default gateway is wrong, or it is on a different VLAN than you think.

Task 3: Read the pcap on OPNsense

tcpdump -r reads a pcap file without re-capturing. This is the operator’s first look at the evidence:

tcpdump -nn -r /root/cap_lan.pcap | head -30

A healthy capture looks like:

12:00:00.000 IP 10.10.10.100.51234 > 93.184.216.34.80: Flags [S], seq 1000
12:00:00.123 IP 93.184.216.34.80 > 10.10.10.100.51234: Flags [S.], seq 2000, ack 1001
12:00:00.123 IP 10.10.10.100.51234 > 93.184.216.34.80: Flags [.], ack 2001
...

If the capture is empty, re-check the interface name and the host IP. If the capture shows the SYN going out but no reply, the firewall is dropping the return or the upstream is not answering — move on to capturing on WAN.

Task 4: Capture on WAN — confirm the source address is translated

The interface here is em1. The point of the capture is to prove outbound NAT is working — the source IP on the WAN side must be OPNsense’s WAN address, not the LAN host’s address.

timeout 30 tcpdump -i em1 -nn -s 0 -w /root/cap_wan.pcap \
    'host 93.184.216.34 and port 80'

From the LAN host:

curl -m 5 http://example.com/ 2>&1 | head -1

Read the capture:

tcpdump -nn -r /root/cap_wan.pcap | head -10

The expected output:

12:00:00.000 IP 198.51.100.1.51234 > 93.184.216.34.80: Flags [S], seq 1000
12:00:00.123 IP 93.184.216.34.80 > 198.51.100.1.51234: Flags [S.], ...

The source address is 198.51.100.1 (OPNsense’s WAN), not 10.10.10.100. If the source is still 10.10.10.100, outbound NAT is broken — the LAN host is reaching the WAN untranslated, which is both a configuration error and a security issue.

Task 5: Diagnose a broken NAT reflection rule

NAT reflection (hairpin NAT) is the case where a LAN host tries to reach a public IP that is OPNsense’s own WAN address. The firewall must translate the destination back to the internal server. Configure the failure mode:

  1. Firewall → NAT → Port Forward: forward external TCP/80 on 198.51.100.1 to internal 10.10.10.50. NAT reflection set to Disable (the failure mode).
  2. Firewall → Rules → WAN: allow TCP/80 to OPNsense WAN IP.
  3. Firewall → Rules → LAN: allow TCP/80 to 10.10.10.50.

From a LAN host:

curl -m 5 -v http://198.51.100.1/ 2>&1 | head -20

The expected symptom: connection times out or connection refused. Capture on three interfaces in parallel:

# In three shells, run each
timeout 15 tcpdump -i em0 -nn -s 0 -w /root/cap_natref_lan.pcap \
    'host 198.51.100.1 and port 80'

timeout 15 tcpdump -i lo0 -nn -s 0 -w /root/cap_natref_lo.pcap \
    'host 198.51.100.1 and port 80'

timeout 15 tcpdump -i em1 -nn -s 0 -w /root/cap_natref_wan.pcap \
    'host 198.51.100.1 and port 80'

Reproduce the failing curl from the LAN host during the captures. Read each:

# Capture on LAN: shows the host's SYN to 198.51.100.1:80 going out
tcpdump -nn -r /root/cap_natref_lan.pcap

# Capture on loopback: shows whether the packet reached OPNsense's lo0
tcpdump -nn -r /root/cap_natref_lo.pcap

# Capture on WAN: shows nothing — the SYN was supposed to hairpin back
# through OPNsense but did not.
tcpdump -nn -r /root/cap_natref_wan.pcap

The failure signature is: SYN on LAN, no SYN-ACK on any interface. NAT reflection is off; the firewall has no rule that translates the destination back to 10.10.10.50.

The fix is NAT reflection = Enable in the port forward rule, or a separate split-DNS host override that resolves the public hostname to the internal IP. Both work; they have different operational trade-offs.

Task 6: Diagnose a broken firewall rule on return traffic

Configure a deliberate failure: a LAN rule that allows outbound TCP/80 but blocks the return traffic because of a typo’d destination.

  1. Firewall → Rules → LAN: add a rule:
    • Action: pass
    • Source: LAN net
    • Destination: 10.10.10.50 (typo; should be any)

The rule allows LAN hosts to reach 10.10.10.50 only. Anything else is dropped by the default deny.

From a LAN host, attempt to reach the internet:

curl -m 10 -v http://example.com/ 2>&1 | tail -10

Capture on LAN:

timeout 15 tcpdump -i em0 -nn -s 0 -w /root/cap_rule_lan.pcap \
    'host 10.10.10.100'

Read the capture:

tcpdump -nn -r /root/cap_rule_lan.pcap

The signature of a stateful firewall drop:

  • The host sends SYN.
  • The reply SYN-ACK never arrives on the LAN interface.
  • OPNsense’s Firewall → Log Files → Live View shows a block on the LAN interface for the return packet, because no state matches it (the original rule had a narrow destination).

The lesson: stateful firewalls drop new packets that match no existing state. A pass rule for outbound traffic also needs to allow return traffic, which stateful inspection handles automatically — unless a rule somewhere strips the state, or the rule is so narrow that the kernel cannot match the return.

Task 7: Diagnose a host on the wrong VLAN

The third scenario: a host that the operator believes is on VLAN 20 is silently on VLAN 10. DHCP gave it an address in the wrong subnet, and the operator is chasing a routing problem that does not exist.

The capture is on the VLAN parent interface with a VLAN tag filter:

timeout 30 tcpdump -i em0 -nn -e -s 0 -w /root/cap_vlan.pcap \
    'vlan 20 and host 10.10.20.100'

The -e flag prints the 802.1Q tag in each line. The output:

12:00:00.000 aa:bb:cc:dd:ee:50 > 01:00:0c:cc:cc:cc, ethertype 802.1Q (0x8100), \
  vlan 20, IP 10.10.20.100.51234 > 10.10.20.1.67: DHCP Discover

If the host is correctly on VLAN 20, the capture shows packets with vlan 20 tagged. If the capture is empty, the host is not on VLAN 20 at all — it may have a misconfigured switch port, an incorrect NIC driver, or be plugged into the wrong physical port.

A second capture on the parent interface without the VLAN filter:

timeout 15 tcpdump -i em0 -nn -e -s 0 -c 20

This shows every VLAN-tagged frame that crosses the parent. If the host in question appears with vlan 10 instead of vlan 20, the operator’s mental model is wrong — the host is on VLAN 10, not 20.

Task 8: Pull the pcaps off OPNsense

The pcaps are useful evidence for an incident report. Pull them with scp:

# From a workstation, not OPNsense
scp root@10.10.10.1:/root/cap_*.pcap ./evidence/

Or use OPNsense’s Diagnostics → Command Prompt → Download to fetch individual files via the GUI.

Do not leave the pcaps in /root indefinitely. They may contain credentials in plaintext (HTTP Authorization headers, LDAP bind passwords, SIP signalling) and they fill the disk over time. Move them to a secured evidence store and delete the originals.

Task 9: Annotate the evidence

For each scenario, write a one-page report. The format:

Scenario: NAT reflection disabled
Symptom: curl http://198.51.100.1/ from a LAN host times out.
Layer: 4 (transport)
Evidence:
  - cap_natref_lan.pcap: 10.10.10.100.51234 → 198.51.100.1.80 [S]
  - cap_natref_lo.pcap: empty
  - cap_natref_wan.pcap: empty
Likely cause: NAT reflection is disabled. The LAN host's SYN
  reached OPNsense, but the firewall had no rule to translate
  198.51.100.1 back to 10.10.10.50.
Remediation: enable NAT reflection on the port forward rule,
  or add a split-DNS host override so the LAN host resolves
  the public hostname to the internal IP.

This is the format the incident-response runbook expects.

Task 10: Clean up the captures

rm -f /root/cap_*.pcap
ls /root/*.pcap 2>/dev/null || echo "no pcaps left in /root"

If you changed any firewall or NAT rules during the lab, undo them now. The lab’s failure-mode rules must not survive the end of the lab.

Validation

  • cap_lan.pcap shows the LAN host’s traffic to a public target, captured on em0.
  • cap_wan.pcap shows the same flow with the source address translated to OPNsense’s WAN IP.
  • cap_natref_lan.pcap shows the SYN going out; cap_natref_wan.pcap is empty — proving the hairpin did not happen.
  • cap_vlan.pcap either confirms or contradicts the host’s VLAN membership, with -e showing the 802.1Q tag.
  • Each scenario has a one-page report in the symptom → layer → evidence → likely cause format.

Expected Result

Three diagnostics, each with a captured pcap and a one-line conclusion. The operator can pick the right OPNsense interface to capture on, run tcpdump with the right filters, and read the result without a GUI.

Troubleshooting

The capture shows nothing. Confirm the interface name with ifconfig -l. The trap is that OPNsense names the interfaces after the driver; em0 is the LAN on one install, the WAN on another.

The capture shows the host’s traffic with the wrong source IP. NAT is failing. Check Firewall → NAT → Outbound for an Automatic rule that covers the LAN subnet, or a manual rule that does the same.

The capture fills the disk. You wrote the pcap to a small filesystem. Use /tmp or rotate files (-W 5 -C 100 for five files of 100 MB each). Stop the capture explicitly with Ctrl-C; do not leave it running.

The capture has the right packets but the GUI is locked. A long-running capture on the GUI’s Diagnostics → Packet Capture page can freeze the session. Use the shell tcpdump from SSH instead; it is more reliable.

Wireshark on the workstation cannot open the pcap. OPNsense’s tcpdump writes pcap-NG by default in newer versions, not pcap. Wireshark handles both, but if you have an old version, specify -U for unbuffered output or convert with editcap cap.pcap cap.pcap.

Cleanup

The lab deliberately broke NAT reflection and tightened a firewall rule. Put both back.

# GUI: revert the port forward rule's NAT reflection to default
# GUI: delete the over-narrow LAN rule from Task 6
# GUI: confirm the LAN rule set is back to the production shape

# Confirm
pfctl -sr | grep -A2 'em0'
pfctl -sn | grep '198.51.100.1'

# Remove the captures
rm -f /root/cap_*.pcap

# Confirm no leftovers
ls /root/cap_* 2>/dev/null || echo "no captures left"

If a host was deliberately placed on the wrong VLAN for the diagnosis, move it back. The lab’s misconfiguration must not survive the end of the lab.

What you learned

  • The OPNsense interface name is not the GUI label. Confirm with ifconfig -l before capturing.
  • A capture on the wrong interface shows nothing and proves nothing. Pick the interface based on the question: LAN capture for “is the host generating traffic?”, WAN capture for “is NAT translating?”, loopback capture for “did the packet reach OPNsense at all?”, VLAN-tagged capture for “is the host really on the right VLAN?”.
  • The failure signatures are different: NAT reflection drops at the destination, firewall drop drops at the state check, wrong-VLAN drops at the switch port. Each has its own pcap fingerprint.
  • A one-page report per scenario, in the symptom → layer → evidence → likely cause format, is the smallest unit of useful incident evidence.

Deliverables

  • · pcap files for each diagnosis scenario, saved on OPNsense
  • · Annotated text output for each pcap, showing the failure signature
  • · A short report covering symptom → layer → evidence → likely cause

Verification status

Last reviewed
2026-08-14
Executed end to end
not yet run on hardware

The commands and configuration here have been reviewed against the verified software versions, but nobody has run this lab start to finish on a system meeting its prerequisites. Treat the Expected Outcome as the intended result rather than an observed one, and keep the Cleanup section to hand.