Routing tables and the control plane

Every OPNsense firewall makes a routing decision on every packet it forwards. The operator who can read the routing table fluently diagnoses most routing incidents in seconds; the operator who cannot read it spends hours clicking through the GUI looking for a checkbox.

The routing table is also where the “control plane” meets the “forwarding plane” — and the two are not always in agreement. This lesson covers how OPNsense builds its routing table, how FreeBSD turns that table into the structure the kernel uses to move packets, and the production traps that follow when the two views diverge.

Control plane, data plane, forwarding plane

Three terms that get used interchangeably but mean different things:

• Control plane is the decision: “for destination 203.0.113.0/24, send to gateway 198.51.100.1 via interface igb0”. Routing protocols (OSPF, BGP, RIP), static configuration, and connected interface setup all live in the control plane.
• RIB (Routing Information Base) is the table the control plane produces. On FreeBSD it is what netstat -rn and route -n show print. The RIB may contain multiple candidate routes for the same prefix (from different sources); the router picks the best one.
• FIB (Forwarding Information Base) is the cache the kernel actually uses on every packet. On FreeBSD it is the structure the route add command writes to and the lookup the IP input path consults. There is normally one entry per prefix — the active candidate from the RIB.

The FreeBSD route command writes to the kernel FIB; netstat -rn shows the FIB. The RIB lives at a higher layer. On OPNsense the GUI shows a third view: the configured static routes, which are loaded into the FIB at boot and on apply.

$ netstat -rn

Routing tables

Internet:
Destination        Gateway            Flags    Refs      Use   Netif Expire
default            198.51.100.1       UGS         0        0     igb1
127.0.0.1          link#4             UH          0        0     lo0
192.0.2.0/24       link#1             U           0        0     igb0
192.0.2.10         link#1             UHS         0        0     lo0
198.51.100.0/24    link#2             U           0        0     igb1
198.51.100.5       link#2             UHS         0        0     lo0

Internet6:
Destination                       Gateway                       Flags      Netif Expire
::1                               link#4                        UHS         lo0
fe80::%igb0/64                    link#1                        U           igb0
fe80::%igb1/64                    link#2                        U           igb1

Where do routes come from?

On OPNsense, three sources can put a route into the FIB:

1. Connected routes. When an interface is configured with an IP and a subnet, FreeBSD installs a connected route automatically. 192.0.2.0/24 on igb0 becomes a route to that subnet via igb0. You cannot delete these from the GUI; they are removed by reconfiguring or disabling the interface.
2. Static routes. Configured at System → Routes → Configuration. Persisted in the OPNsense config XML and loaded by the route add command at boot and on apply.
3. Dynamic routes. Pushed by routing daemons: FRR (the OPNsense bundled fork of Quagga) for OSPF, BGP, RIP, IS-IS; or BIRD if installed separately. Dynamic routes appear in the kernel FIB the moment the daemon announces them.

Each source has a preference (sometimes called “administrative distance” on other platforms; FreeBSD has no single global “distance” concept but per-source precedence rules apply). Connected routes win over static, static wins over dynamic by default. The control plane may know about multiple equal candidates — the FIB holds only the active one.

How to read what the kernel is actually doing

netstat -rn is the canonical FreeBSD command. Two flags matter most for production diagnostics:

• -r — show the routing tables.
• -n — show numeric addresses, no DNS resolution. Without -n the command can hang on DNS lookups for unreachable destinations.

For a specific destination, route -n get <destination> is the fastest way to see which route the kernel picks and why:

$ route -n get 203.0.113.50

   route to: 203.0.113.50
destination: 0.0.0.0
     mask: 0.0.0.0
  gateway: 198.51.100.1
interface: igb1
    flags: <UP,GATEWAY,DONE>
recvpipe  sendpipe  ssthresh  rtt,msec    mtu        expire
 0         0         0         0         1500        0

The output shows gateway: 198.51.100.1 and interface: igb1 — that is the FIB decision the kernel will make. If pfctl -s state later shows traffic for that destination on a different interface, you have asymmetric routing, even if the FIB says it should be on igb1. The FIB and the live traffic path can diverge when routing changes mid-flow (the kernel keeps state on the old interface for the lifetime of the connection).

Common production incidents

Three failure modes show up over and over:

1. “The route is gone after a reboot” — a static route that was added by hand at the shell (route add ...) does not persist; OPNsense rebuilds the FIB from the config XML at boot. The fix is to add the route through the GUI (System → Routes → Configuration) or the API.
2. “FRR changed the default route” — see the callout above. The FIB has a different default than the GUI shows because the dynamic protocol won the preference race.
3. “The route is right, the packet still goes the wrong way” — this is almost always an existing state. PF holds a flow on the interface the SYN arrived on. The FIB may have changed; the existing connection is bound to the old egress. Wait for the state to expire or clear it explicitly with pfctl -k state.

Summary

• The control plane decides. The RIB records what was decided. The FIB is what the kernel uses on the fast path. They can diverge.
• On OPNsense, routes come from connected interfaces, static configuration, and routing daemons (FRR).
• netstat -rn shows the FIB. route -n show shows the same data in a different format. route -n get <dest> shows what the kernel will do for one specific destination.
• A “missing” route is often a route hidden behind a more-preferred one, not a route that was never installed.