Source to ingress interface — what happens before PF sees the packet

The packet flow starts before the firewall. The source host sends the packet; the switch forwards it; ARP resolves the next-hop MAC; the frame arrives on the firewall’s ingress interface. The operator who starts diagnosing at PF without verifying the ingress is missing half the problem. Sometimes the packet never arrives at the firewall.

This lesson covers the path from source to ingress, the role of the switch and VLAN tags, the symptoms of switch-level drops, and the discipline of verifying ingress before assuming PF involvement.

The pre-firewall path

A packet from a LAN host destined for an Internet server traverses this path before the firewall sees it:

[Host A]
  → [Host A's IP stack: route table lookup]
  → [ARP: next-hop IP → MAC]
  → [NIC: frame transmission]
  → [Switch: learn source MAC, forward by destination MAC]
  → [Cable / fibre to firewall]
  → [Firewall NIC: frame reception]
  → [NIC driver: DMA to kernel memory]
  → [Ingress interface in OPNsense]

At the firewall ingress interface, the frame has been fully transmitted by the host, fully forwarded by the switch, and fully received by the firewall NIC. The kernel has the frame in memory; BPF has copied it (if a capture is running); PF has not yet evaluated it.

The discipline: verify the packet is at the firewall before assuming PF involvement.

The role of the switch

The switch is the silent partner in the pre-firewall path. The switch:

• Learns source MACs. When a frame arrives on a port, the switch records the source MAC in its MAC table, associated with that port.
• Forwards by destination MAC. When the switch has an entry for the destination MAC, it forwards the frame only to that port. When it does not, it floods the frame to all ports.
• Handles VLAN tags. On a trunk port, the switch adds or removes 802.1Q tags as configured.

If the switch is misconfigured:

• Frames are dropped at the switch. The host sends the frame; the switch does not forward it to the firewall’s port. The firewall never sees the packet.
• Frames are flooded. The switch has no MAC entry and floods the frame. Performance suffers, but the firewall still sees the frame.
• VLAN tags are wrong. The frame arrives on the firewall with an unexpected tag — the firewall’s VLAN classifier drops it.

$ tcpdump -ni igb0 -e -c 1 arp

12:34:56.789012 aa:bb:cc:11:22:33 > ff:ff:ff:ff:ff:ff, ARP, Request who-has 192.0.2.1 tell 192.0.2.50, length 28

ARP resolution

Before the host can send the IP packet, it must know the next-hop MAC. The host looks up the next-hop IP in its routing table (the default gateway for the LAN host is the firewall’s LAN IP). If the host has no ARP entry for the next hop, it broadcasts an ARP request and waits for the reply.

If the ARP reply never arrives:

• The switch dropped the request or reply. The broadcast was flooded but the firewall’s port was not in the same VLAN.
• The firewall is not answering ARP. The firewall’s interface is down, misconfigured, or the host is on the wrong subnet.
• The host’s ARP table has a stale entry. The host has an ARP entry for the next hop but the MAC is wrong.

The discipline is to check the firewall’s ARP table when a host cannot reach the firewall:

arp -an

The operator confirms the firewall has the host’s MAC in its ARP table. If not, the firewall cannot send frames to the host — the problem is at Layer 2, not at PF.

VLAN tags in the pre-firewall path

On a trunk port, frames carry 802.1Q tags identifying the VLAN. The switch adds the tag when sending a frame on a trunk; the firewall removes the tag when receiving on a VLAN interface.

If the tags are wrong:

• Frames arrive with no tag on a trunk port. The firewall treats them as untagged traffic on the native VLAN (usually VLAN 1).
• Frames arrive with the wrong VLAN tag. The firewall drops them — the VLAN ID does not match any configured VLAN interface.
• Frames arrive with a tag the firewall is not configured for. The kernel drops them at the VLAN classifier; they never reach PF.

The operator verifies VLAN configuration with ifconfig on the firewall:

ifconfig igb0_vlan10

The output shows the VLAN ID, parent interface, and IP address. The operator confirms the VLAN ID matches what the switch is sending.

Capturing on ingress to verify arrival

The first diagnostic step for “the host cannot reach X” is to capture on the firewall’s ingress interface. The filter targets the host’s IP and the destination:

tcpdump -ni igb0 host 192.0.2.50 and host 203.0.113.5

If the capture shows the packet, the firewall saw it. If the capture shows nothing:

• The host did not send it. Check the host’s routing table, default gateway, and ARP cache.
• The switch dropped it. Check the switch’s MAC table, port status, and VLAN configuration.
• The firewall NIC did not receive it. Check the cable, NIC status (ifconfig igb0), and link lights.

The operator who starts at PF without verifying ingress may waste hours chasing a problem that does not exist.

$ tcpdump -ni igb0 'host 192.0.2.50' -c 5

12:34:56.789012 ... 192.0.2.50.51820 > 192.0.2.1.51820: ICMP echo request, id 1234, seq 1, length 64
12:34:57.789012 ... 192.0.2.50.51820 > 192.0.2.1.51820: ICMP echo request, id 1234, seq 2, length 64
...

Symptoms of pre-firewall drops

The signatures of pre-firewall problems:

• No packets on ingress capture. The host is not sending, the switch is not forwarding, or the firewall NIC is not receiving. Diagnose in that order.
• ARP requests but no replies. The firewall is not answering ARP. Check the interface state, IP configuration, and VLAN membership.
• Frames with unexpected VLAN tags. The switch is tagging traffic for a VLAN the firewall is not configured for. Reconcile the switch and firewall VLAN configuration.
• Frames with no tag on a trunk port. The switch is sending untagged traffic; the firewall treats it as the native VLAN.

The trap is to assume the firewall is at fault when the problem is upstream. The operator who checks PF first and finds no block rule concludes the firewall is fine, then walks away — leaving the real problem (a switch port in the wrong VLAN) unresolved.

The pre-firewall checklist

Before assuming PF is involved, the operator should verify:

1. The host is up and has IP connectivity to the firewall. Ping the firewall from the host.
2. The host has the firewall as its default gateway. Check the host’s routing table.
3. The host has an ARP entry for the firewall. arp -an on the host.
4. The firewall has an ARP entry for the host. arp -an on the firewall.
5. The switch port is up and in the correct VLAN. Check the switch’s port status.
6. The firewall interface is up and configured. ifconfig shows the interface as status: active.
7. A capture on the firewall ingress shows the packet.

If all seven are true, the packet is at the firewall and PF is the next suspect. If any is false, the problem is upstream and PF is irrelevant.

Summary

• The pre-firewall path: host IP stack → ARP → NIC → switch → firewall NIC → kernel.
• The switch silently forwards frames by MAC; misconfiguration at the switch causes drops the firewall never sees.
• ARP must succeed before IP packets can flow; ARP failure looks like a firewall problem from above.
• VLAN tags in the pre-firewall path must match between the switch and the firewall; mismatched tags are dropped at the kernel VLAN classifier.
• The first diagnostic step is to capture on the firewall ingress interface; if no packet is there, the problem is upstream.
• The pre-firewall checklist: host connectivity, gateway, ARP on host, ARP on firewall, switch port, firewall interface, ingress capture.