Runbook: Troubleshoot Outbound NAT

Purpose

This runbook diagnoses and fixes outbound NAT issues: wrong egress source IPs, missing outbound NAT rules that cause upstream drops, and the multi-WAN case where traffic leaves the wrong gateway and gets blackholed by asymmetric routing.

When to use this runbook

Use this runbook when internal hosts cannot reach the internet (or reach it with the wrong source IP) and the firewall is suspected. It is also the right starting point when a multi-WAN deployment is failing over correctly at the gateway layer but traffic is still emerging from the wrong WAN.

Pre-checks

• Capture a fresh encrypted config backup before any change.
• Open Firewall > NAT > Outbound and note the current mode: Automatic, Manual, or Hybrid.
• Confirm the issue shape: is the egress source IP wrong (some other internal IP, or an RFC1918 address), or is the flow not leaving at all?

Procedure

1. Pick the right mode. Open Firewall > NAT > Outbound. Automatic is the default and works for simple single-WAN environments. Hybrid is recommended for production: it auto-generates rules for interface subnets and lets you add custom rules below. Manual gives full control but is error-prone.
2. In Hybrid mode, regenerate the auto rules. Click Generate rules to recreate the per-interface subnet-to-WAN mapping. Add custom rules below for source-specific mappings (multi-WAN failover, source-to-IP pinning, RFC1918 leak prevention).
3. Confirm source coverage. For every internal subnet that needs internet, there must be a NAT rule that maps it to a WAN address. If a subnet is missing, OPNsense leaves the source IP intact, and many upstream routers will drop RFC1918 sources.
4. Inspect the compiled NAT table. At Diagnostics > Command Prompt, run pfctl -sn | head -40 to see the actual NAT rules OPNsense is running. Confirm there is a rule mapping the affected subnet to the WAN.
5. Test egress source. From an internal host, run curl -4 https://ifconfig.me or curl -4 https://api.ipify.org and note the egress IP. Compare it to the WAN IP at OPNsense (ifconfig <wan_if> at the shell).
6. Interpret the result. If the egress IP is the WAN IP, outbound NAT is working. If it is an internal IP or another WAN’s IP, a NAT rule is missing or outranks the auto rule. If the request times out, the flow is not leaving at all — likely a firewall or routing issue, not NAT.
7. Add a Hybrid NAT rule if missing. Click + to add a rule: set Interface to the WAN, Source to the internal subnet or alias, Translation / target to WAN address, Translation / port as appropriate. Save and apply.
8. For multi-WAN, align NAT with routing. The NAT rule must reference the same WAN interface as the route-to directive in the matching firewall rule, and the route must come from the same gateway group. If NAT and routing disagree, traffic leaves one WAN but the source IP is rewritten to another WAN — the upstream router blackholes it.
9. Test failover. On multi-WAN, disconnect WAN1 (or set the WAN1 gateway to Disabled in System > Gateways) and re-run the egress test from the internal host. Within the gateway monitoring interval (default ~10s for apinger), the egress IP should switch to WAN2.
10. Capture on the WAN interface if traffic is not leaving. Run a packet capture on the WAN interface filtered to the source IP and destination IP. If OPNsense is not generating packets, the issue is upstream of NAT — most likely a firewall rule blocking the outbound flow.

Verification

• curl -4 https://ifconfig.me from the internal host returns the expected WAN IP.
• On multi-WAN, failing over WAN1 flips the egress IP to WAN2 within the gateway monitoring interval.
• pfctl -sn | grep <internal_subnet> shows the NAT rule mapping the subnet to the correct WAN.
• Destination services (HTTPS, ICMP, application flows) succeed from the internal host end-to-end.

Rollback

• Disable any custom Hybrid NAT rules that were added; click Generate rules again to restore the automatic set.
• If the mode was changed, switch back to the prior mode and apply.
• Restore the pre-change encrypted config backup if alias edits were part of the change.
• Verify egress returns to the prior state with curl -4 https://ifconfig.me.

Escalation

• Escalate to the network team if the upstream router is filtering source IPs and blocking RFC1918 egress; some ISPs and cloud peering reject unmapped sources.
• Escalate to security if the request is to NAT traffic to a different source IP than the WAN address; document the business reason and check with the change record owner.
• Escalate to the application owner if the application is binding to a specific source IP that OPNsense cannot rewrite (rare; some kernel sockets set IP_FREEBIND).
• Open an incident if outbound NAT failure breaks customer-facing traffic for longer than the agreed SLA window.