Interface mapping considerations across hardware revisions

A backup that was taken from appliance A does not always restore cleanly onto appliance B. The XML references interfaces by FreeBSD driver name — igb0, ix0, vtnet1 — and the operator who copies a backup between two appliances with different NIC layouts discovers, at boot time, that the WAN interface is now on the LAN port and the LAN interface is on the WAN port. The firewall comes up, the routes look right, the NAT rules look right, and the network silently misroutes every packet.

This lesson covers the NIC ordering trap, the sections of the XML that are sensitive to interface assignment changes, the test procedure that catches the mismatch before production, and the migration path when a backup must be adapted rather than restored as-is.

The NIC ordering trap

OPNsense references interfaces in the XML by their FreeBSD driver name. The driver’s enumeration order is determined at boot by the PCI bus scan and the driver initialisation order, both of which can change between hardware revisions even for the same vendor.

A backup taken from an appliance with:

igb0 = WAN
igb1 = LAN
igb2 = DMZ
igb3 = SYNC (CARP)

Restored to an appliance with a different NIC order:

igb0 = LAN  ← was WAN in the backup
igb1 = DMZ  ← was LAN in the backup
igb2 = SYNC ← was DMZ in the backup
igb3 = WAN  ← was SYNC in the backup

The XML is unchanged. The interface assignments are now wrong. The firewall applies the WAN rules to the LAN interface, blocks DMZ traffic with the SYNC rules, and exposes the SYNC interface to the Internet with the WAN rules. The boot succeeds. The GUI is reachable. The disaster is silent.

What in the XML is sensitive to interface order

Most of the XML cares only about logical names (wan, lan, dmz, sync). The XML does not directly reference driver names — the operator-assigned names are mapped to driver names through the <interfaces> section and the GUI’s interface assignment page. Three things, however, are sensitive to the underlying driver order:

The operator who restores a backup to different hardware must remap the <if> entries from the source driver names to the destination driver names. The remap is straightforward when the destination has the same NIC model; it is error-prone when the destination has a different chipset or count.

$ pciconf -l | grep -i ethernet

em0@pci0:0:25:0:    class=0x020000 rev=0x03 hdr=0x00 vendor=0x8086 device=0x1502
em1@pci0:0:26:0:    class=0x020000 rev=0x03 hdr=0x00 vendor=0x8086 device=0x1502
igb0@pci0:1:0:0:    class=0x020000 rev=0x01 hdr=0x00 vendor=0x8086 device=0x1521
igb1@pci0:1:0:1:    class=0x020000 rev=0x01 hdr=0x00 vendor=0x8086 device=0x1521
igb2@pci0:1:0:2:    class=0x020000 rev=0x01 hdr=0x00 vendor=0x8086 device=0x1521
igb3@pci0:1:0:3:    class=0x020000 rev=0x01 hdr=0x00 vendor=0x8086 device=0x1521

MAC address drift

Each interface has a MAC address, and OPNsense stores the MAC of each interface in the configuration. The MAC is used for:

• CARP advertisements. A CARP VIP has a virtual MAC (00:00:5e:00:01:<vhid>) that is independent of the physical interface MAC. Restoring a backup does not affect the CARP MAC.
• Bridge member interfaces. The bridge is identified by MAC, and bridge member interfaces are tracked by MAC. A backup from appliance A has bridge MACs from appliance A; restoring to appliance B creates a new bridge with appliance B’s MACs. The bridge stays up but its MAC changes.
• HA hardware failover. Some HA configurations track the primary appliance MAC and fail over if the MAC changes. Restoring a backup to a replacement appliance changes the MAC and may trigger an unplanned failover.
• DHCP client identity on the WAN. Some ISPs bind the DHCP lease to the MAC of the CPE. A backup that changes the WAN MAC triggers a new DHCP request that may take hours to converge.

The disciplined pattern is to document the MAC addresses of every interface at provisioning time. The MAC list is part of the appliance’s identity record. After a restore, the MACs are verified against the record. A mismatch is the signal to investigate before the production traffic flows.

The test procedure

Before a cross-hardware restore reaches production, the operator runs a four-step test:

1. Stand up the destination hardware in isolation. Boot OPNsense on the destination, complete the first-boot wizard, and note the driver order (pciconf -l, ifconfig).
2. Apply the backup in a non-production mode. Restore the XML but boot into single-user mode first. Inspect the interface assignment file and confirm the driver mappings are correct.
3. Bring up the interfaces in sequence. Bring up each interface with ifconfig <if> up and confirm the link light on the physical switch port matches the expected logical name.
4. Smoke-test with a single host. Plug a single laptop into the LAN port of the destination, confirm it gets a DHCP lease, can reach the GUI, and can ping an external address. Only then connect the destination to the production network.

The test catches the NIC ordering trap and the MAC drift before the production firewall is misrouted. The test takes 15 minutes. The misroute takes hours to diagnose.

When to migrate the backup instead of restoring as-is

Some cross-hardware migrations require more than a remap:

• Different NIC count. The source has 4 interfaces, the destination has 6. The backup assigns WAN/LAN/DMZ/SYNC; the destination has two free ports. Migration adds the new ports, not just remaps.
• Different interface roles. The source uses igb3 for a CARP sync interface; the destination uses a dedicated 1G fibre port. Migration renames the role and reassigns.
• Different VLAN parent. The source trunks VLAN 10 on igb0; the destination trunks VLAN 10 on bge0. Migration rewrites the VLAN parent.
• Different driver family. The source uses igb; the destination uses ix or vtnet. The XML entries must be rewritten to the new driver names.

In each case, the disciplined path is to apply the backup, then walk through the Interfaces → Assignments page to remap, then validate each VLAN, each bridge, and each IPsec tunnel that references the changed interfaces. The migration is not a single-step restore. It is a restore plus a reassignment plus a validation.

Summary

• NIC ordering is not stable across hardware revisions. A backup assumes an order that the destination may not have.
• The XML’s <if> entries hold driver names. Cross-hardware restores must remap.
• MAC addresses drift between appliances. Document them. Verify after restore.
• The four-step test catches the misroute before production.
• Some migrations need reassignment, not just restore.