Interface tracking and gateway failover · OPNsense

A firewall with two WAN links must decide, on every outbound packet, which WAN to use. When one WAN fails, the firewall must detect the failure quickly, switch to the other WAN, and ideally bring rules on the failed interface into a sensible state. This lesson covers interface tracking (how OPNsense monitors gateway health), gateway groups (how failover is expressed in PF), and the discipline of disabling rules on a failed interface.

What interface tracking is

Interface tracking is OPNsense’s mechanism for monitoring the health of a gateway. The firewall periodically probes the configured gateway IP (e.g. the ISP’s default route) and marks the gateway as up or down based on probe success.

The probe is dpinger, a small daemon that sends ICMP echo requests to the gateway at a configurable interval and measures the loss. By default, OPNsense probes every 1 second and considers a gateway down after 5 consecutive failures (roughly 5 seconds of unreachability).

When a tracked gateway is down:

The interface’s default route is removed from the FIB.
Any gateway group that included this gateway marks it as failed and the group falls over to the next tier.
Rules that reference the gateway group re-evaluate.

Read-only / Safedpinger output— dpinger prints one line per probe round. Each line shows the gateway IP, the round-trip latency, and the packet loss percentage. Loss >0% for several consecutive rounds triggers the gateway to be marked down. Loss = 0% and consistent latency means the gateway is healthy. The output is typically written to the system log at System → Log Files → General.

$ dpinger -S -r 0 -i WAN_DHCP -B 198.51.100.1 -p /var/run/dpinger_WAN_DHCP.pid -u /var/run/dpinger_WAN_DHCP.sock

dpinger: GW 198.51.100.1, latency 12.4ms, loss 0.0%
dpinger: GW 198.51.100.1, latency 11.8ms, loss 0.0%
dpinger: GW 198.51.100.1, latency 13.1ms, loss 0.0%

Illustrative output

Gateway groups

A gateway group is a logical construct that ranks multiple gateways by tier. A common production design:

Tier 1: primary ISP (e.g. fibre).
Tier 2: secondary ISP (e.g. cellular failover).

When both gateways are up, traffic uses Tier 1. When Tier 1 fails, traffic fails over to Tier 2. When Tier 1 recovers, traffic returns to Tier 1 (with a configurable hysteresis to prevent flapping).

Gateway groups are configured at System → Gateways → Groups. Each group has a name, a tier for each member gateway, and a “trigger level” — what level of degradation triggers failover.

A rule references a gateway group by name. The rule is then applied only when the gateway group has a member at the relevant tier. When the tier’s only gateway is down, the rule is suspended — the rule does not match until the gateway recovers.

Disabling rules on a failed interface

The next question is: when the interface (not just the gateway) goes down — the WAN cable is unplugged, the ISP’s edge router is powered off, the cellular modem has lost its connection — should the firewall’s rules on that interface still be considered?

OPNsense has an option called “Disable gateway monitoring” or “Skip rules when gateway is down”. When enabled:

If the gateway for an interface is down, PF rules with gateway <group_name> are skipped.
The rule does not match. New connections are not permitted on the failed interface’s path.

This is exactly the right behaviour for a gateway group with failover: the rule’s intent is “use ISP-A when ISP-A is up, else use ISP-B”. When ISP-A’s gateway is down, the rule skips and the next tier takes over.

The configuration is per-rule: under the “Advanced features” section of the rule editor, there is a checkbox for “Skip rules when gateway is down”. This binds the rule’s behaviour to the gateway state.

The discipline of tracking

Interface tracking should be enabled for every interface that has an upstream gateway — WAN interfaces, DMZ interfaces with upstream firewalls, and so on. The settings to check:

Monitor IP: the IP dpinger probes. For a WAN with a DHCP- assigned gateway, this is set automatically. For a static configuration, the operator enters the upstream router’s IP.
Probe interval: how often dpinger probes. Default 1 second is appropriate for most setups; 100 ms is appropriate for high-availability requirements.
Loss threshold: how many consecutive failures mark the gateway down. Default 5 is appropriate; lower values produce faster failover at the cost of more false positives.
Down time after recovery: hysteresis. After the gateway recovers, wait this long before considering it stable. Default is 0 seconds; production setups often use 30–60 seconds to prevent flapping.

Read-only / Safepfctl states— Active state table entries showing traffic to both ISP gateways (198.51.100.1 and 203.0.113.1). When one ISP's gateway is marked down, new connections skip that interface; existing connections remain bound to the old route until they expire. The 'SINGLE:NO_TRAFFIC' for UDP states means the state has not seen traffic in a while but is still valid.

$ pfctl -s states | grep -E '198.51.100.1|203.0.113.1' | head -10

igb0 tcp 192.0.2.50:51234 -> 198.51.100.1:443 ESTABLISHED:ESTABLISHED
igb1 tcp 192.0.2.50:51235 -> 203.0.113.1:443 ESTABLISHED:ESTABLISHED
igb0 udp 192.0.2.50:6000 -> 198.51.100.1:53 SINGLE:NO_TRAFFIC
igb1 udp 192.0.2.50:6001 -> 203.0.113.1:53 SINGLE:NO_TRAFFIC

Illustrative output

Diagnosing a tracked interface that is down

When dpinger marks a gateway down, the operator needs to know:

Is the gateway really unreachable? ping <gateway_ip> from the firewall’s diagnostic page.
Is the link up? ifconfig <interface> shows the link status (status: active for up, status: no carrier for down).
What does the firewall’s log show? System → Log Files → General shows dpinger’s transitions: “WAN_DHCP: 198.51.100.1 latency 0ms loss 100%” then “GW 198.51.100.1 down”.
What does the gateway group show? System → Gateways → Groups shows each member’s status and tier.

The fix depends on the cause:

Link down: check cables, SFP modules, switch ports.
Gateway unreachable but link up: ISP issue; the firewall is working correctly, the upstream is broken.
dpinger misconfigured: probe IP is wrong, or the upstream blocks ICMP.

Production patterns

Three patterns cover most decisions:

Single-WAN deployments: enable tracking on WAN with the ISP’s gateway as the monitor IP. No gateway group needed.
Dual-WAN failover: configure both WANs with tracking, build a gateway group with WAN1 at tier 1 and WAN2 at tier 2, reference the group in the LAN rule.
Dual-WAN load balancing: configure both WANs with tracking, build a gateway group with both at tier 1 (load-balanced). Mark the “failover” tier higher to prefer one over the other.

Summary

Interface tracking monitors gateway health via dpinger (ICMP echo at a configurable interval).
A gateway group ranks multiple gateways by tier; traffic fails over from a down tier to the next tier.
“Skip rules when gateway is down” is a per-rule option that suspends the rule when the gateway is down.
Best practice: enable the toggle on every rule that references a gateway group, so rules are skipped when the gateway is down and traffic fails over to the next tier.
Existing PF state stays on the old route until expiry; only new connections follow the new gateway.