Bridges, laggs, and bonding

OPNsense can combine multiple physical interfaces into a single logical interface. Two mechanisms cover most needs: bridges combine interfaces at Layer 2 (the firewall forwards frames based on MAC, like a switch would); laggs combine interfaces at Layer 1 (multiple physical links act as one logical link for throughput and redundancy). This lesson covers both, when to use each, and the failure modes that surface in production.

Bridges — Layer 2 combining

A bridge is a virtual switch: the firewall forwards frames between member interfaces based on MAC addresses. A bridge is the right tool when:

• Multiple interfaces need to be on the same Layer 2 segment.
• The firewall should forward frames between two segments at Layer 2 (not route them).
• The operator needs a single interface that has multiple physical uplinks as fallback.

The classic example: a firewall with two physical NICs going to the same physical switch, used as a redundant pair. A bridge of those two NICs gives a single logical interface that keeps working even if one physical link fails.

OPNsense exposes bridges under Interfaces → Assignments → Bridges. The operator creates a bridge, adds member interfaces, and assigns the bridge as a normal interface. The kernel creates a bridge0 interface with the member NICs as ports.

$ ifconfig bridge0

bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
  options=0
  ether a4:5e:60:11:22:33
  inet 192.0.2.1 netmask 0xffffff00 broadcast 192.0.2.255
  id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
  maxage 20 holdcnt 6 proto rstp
  member: igb2 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP> portif 1 portno 0x1
  member: igb3 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP> portif 2 portno 0x2

Spanning Tree and bridge loops

When two bridges are connected to the same Layer 2 segment with two paths between them, a forwarding loop occurs: a broadcast frame is forwarded by both bridges, then forwarded again, forever, in a loop that saturates the network. Spanning Tree Protocol (STP) prevents this by electing one path as active and blocking the others.

OPNsense bridges run RSTP (Rapid Spanning Tree) by default. The firewall participates in STP with the upstream switches; one of the paths is blocked, the other forwards. If the active path fails, STP converges (typically within seconds) and the backup path takes over.

The production trap: if the operator disables STP on a bridge that has multiple paths to the same Layer 2 segment, a forwarding loop will saturate the network within seconds. STP should be enabled unless the operator has carefully verified there is exactly one path between any two bridge ports.

Laggs — link aggregation

A lagg (FreeBSD term; “bond” on Linux) combines multiple physical links into one logical link. The two main purposes:

• Throughput: traffic is distributed across multiple physical links (typically by hash of MAC or IP). Two 1 GbE links in a lagg can theoretically deliver 2 Gbit/s.
• Redundancy: if one link fails, the lagg continues on the other links.

The modes:

For production, LACP is the right choice. The upstream switch and the firewall negotiate the aggregation; if the operator changes cabling, LACP renegotiates automatically.

OPNsense exposes laggs under Interfaces → Assignments → LAGG. The operator adds member interfaces, selects LACP as the protocol, and assigns the lagg as a normal interface.

Choosing the right aggregation mode

Three patterns cover most decisions:

1. Single-link redundancy: use LACP with two 1 GbE NICs. Throughput up to 2 Gbit/s with multi-flow traffic; redundancy if either link fails. Standard production setup for a 1–2 Gbit/s firewall.

2. Bridge for L2 failover: use a bridge of two NICs going to the same switch. The bridge has one logical IP, but at Layer 2 the kernel forwards frames on whichever NIC reaches the destination MAC. STP prevents loops if the switch also has redundant paths.

3. Multi-chassis lagg (MC-LAG): rare; requires switch support. Two firewalls’ laggs across two switches present a single logical link.

$ ifconfig lagg0

lagg0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
  options=4e527bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWTSO,NETIF_TSO6,LRO6>
  ether a4:5e:60:44:55:66
  inet 192.0.2.1 netmask 0xffffff00 broadcast 192.0.2.255
  laggproto lacp lagghash l2,l3,l4
  laggport: igb2 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
  laggport: igb3 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>

Production pitfalls

Three failure modes are common:

1. LACP mismatch. The firewall is configured for LACP but the switch is statically configured (or vice versa). LACP negotiation fails, the lagg does not come up, traffic stops. Fix: ensure both sides use LACP, or both sides use static configuration.

2. Asymmetric member speed. One member of the lagg is 1 GbE and another is 100 Mbit/s (e.g. a switch port negotiated down due to a bad cable). The slower link becomes the bottleneck for any flow hashed to it. Fix: verify link speed on each member with ifconfig <member> | grep media.

3. STP on the bridge, no STP on the lagg. A bridge needs STP; a lagg does not (because LACP prevents the loop by aggregating into one logical link). Configuring STP on a lagg or omitting it on a bridge produces confusing behaviour. Fix: STP on bridges only.

Production patterns

Three patterns cover most decisions:

1. Single 1 GbE uplink: no aggregation. One NIC, one link.
2. Redundant 1 GbE uplinks: LACP lagg of two NICs. Both active, redundancy on either failure, throughput up to 2 Gbit/s with multi-flow traffic.
3. Multiple LAN segments at Layer 2: bridge across two NICs to span the same IP subnet across two physical switches (typically for failover between data centres or between access switches).

Summary

• Bridges combine interfaces at Layer 2 (MAC forwarding); use when multiple interfaces should be on the same subnet.
• Laggs combine interfaces at Layer 1 (link aggregation); use for throughput or redundancy with LACP.
• LACP negotiates with the upstream switch; static LAG does not.
• Spanning Tree must be enabled on bridges to prevent loops.
• LACP throughput scales with flow count, not with single-flow bandwidth.