CARP virtual IP configuration on OPNsense — the GUI, the ifconfig, and the packet flow · OPNsense

The OPNsense GUI exposes CARP through two pages: Interfaces → Virtual IPs → Add for the virtual IP itself, and System → High Availability → Settings for the CARP daemon, the sync interface, and the synchronisation password. The VIP page is where most of the operational decisions live — VHID, advertising interface, password, skew, frequency. The HA settings page is where the sync (pfsync) interface is named and the XMLRPC remote is configured.

This lesson walks the configuration, maps each GUI field to the kernel state it produces, and traces how a packet flows through the master (and through the backup when the local node is not the master). The lesson is intentionally concrete — the goal is that an operator who has read the lesson can configure a CARP VIP from the GUI and predict what ifconfig will show.

The GUI fields

To add a CARP VIP: Interfaces → Virtual IPs → Add. The form asks for:

Mode — CARP. (Other modes are IP Alias, Proxy ARP, Other. CARP is the only one that participates in HA.)
Interface — the OPNsense interface the VIP will advertise on. Conventionally the LAN, the WAN, or a DMZ. The interface must be up and have a physical IP.
IP address — the shared IP. Must be in the same subnet as the interface’s physical IP.
VHID — the Virtual Host ID, 1 to 255. Each VHID on a segment must be unique.
Password — the CARP authentication password, 1 to 20 characters. Must match exactly on the secondary.
Advertising frequency — the base advertisement interval. Default 1 (one second). The OPNsense GUI exposes this as a base; the skew is a separate field.
Skew — the base advertisement skew. Default 0 on the primary, 100 on the secondary.
Description — free text. Use something that identifies the role and the segment, e.g. lan-vip-primary.

The OPNsense convention is to enable forcible failover by setting the primary’s skew to 0 and the secondary’s skew to 100. Some operators use 10 and 100 to give the primary a small preference while leaving room for the secondary to demote (tie-breaking without a demote gets ugly).

The CARP password is set on the HA settings page, not the VIP page. The password is per-segment, not per-VIP — the same password authenticates every CARP advertisement on the same network. Operators who run multiple CARP segments (LAN, WAN, DMZ) use the same password on each segment, or different passwords if they want to prevent accidental cross-segment authentication.

Read-only / Safeifconfig igb0— The kernel view of the CARP VIP. There is no carp0 device: CARP is a property of the parent interface, so the VIP is queried by inspecting the interface it was configured on. The 'inet 192.0.2.1 ... vhid 1' line is the VIP and the VHID it belongs to; 192.0.2.2 is the node's own address on the segment. The 'carp: MASTER vhid 1' line is the state, the VHID, the base advertisement interval, and the base skew.

$ ifconfig igb0

igb0: flags=8963<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
      ether aa:bb:cc:11:22:44
      inet 192.0.2.2 netmask 0xffffff00 broadcast 192.0.2.255
      inet 192.0.2.1 netmask 0xffffff00 broadcast 192.0.2.255 vhid 1
      media: Ethernet autoselect (1000baseT <full-duplex>)
      status: active
      carp: MASTER vhid 1 advbase 1 advskew 0

Illustrative output

Mapping the GUI to the kernel

Each GUI field has a corresponding kernel state. The OPNsense configuration is stored in config.xml under <virtualip> and <hasync>. The configctl interface list shows the live state:

GUI field	config.xml	Kernel state
Mode = CARP	`<mode>carp</mode>`	CARP enabled for this address
Interface	`<interface>lan</interface>`	the VHID is attached to that parent interface
IP address	`<ipaddr>192.0.2.1/24</ipaddr>`	`inet` alias on the parent, tagged with the `vhid`
VHID	`<vhid>1</vhid>`	`vhid` in the carp: line
Password	`<password>p4ssw0rd</password>`	HMAC key for the VHID
Frequency	`<advbase>1</advbase>`	`advbase` in the carp: line
Skew	`<advskew>0</advskew>`	`advskew` in the carp: line

The ifconfig igb0 output above shows the kernel state produced by the GUI configuration. The mapping is direct — the operator who can read ifconfig can tell whether the GUI took the configuration correctly.

Verify the kernel state after every CARP configuration change

The OPNsense GUI is reliable, but the path from the GUI to the kernel is long — the configuration is XML, the XMLRPC sync copies it to the secondary, the secondary applies it, and the kernel applies the VHID to the parent interface. A typo in the password on the secondary, a VHID collision with another VIP on the same segment, or a mismatched subnet will fail silently — the GUI will accept the change and the kernel will create the VIP, but the CARP election will resolve to a wrong state.

The discipline: after every CARP change, run ifconfig on both nodes and verify the VHID, the password hash, the advbase, the advskew, and the state. The ifconfig output is the source of truth — the GUI is what is configured, the kernel is what is running.

Packet flow when the local node is master

When the local node is the master for a VHID, the kernel is the owner of the VIP. The packet flow for an inbound frame addressed to the VIP’s MAC is:

Frame arrives on the physical interface. The NIC driver delivers the frame to the kernel. The destination MAC is the CARP multicast MAC 01:00:5e:00:00:12 (for advertisements) or the VIP’s MAC (for data frames, which is the master’s MAC).
Kernel decapsulates the frame. The IP packet is handed to the IP layer. The destination IP is the VIP.
IP routing decision. The local routing table matches the destination IP to the local carp interface. The kernel marks the packet as IFF_LOCAL — destined for the local host.
PF input. PF sees the packet as INPUT on the carp interface. The rule set is evaluated. If the rule allows the traffic, the packet is passed.
Local delivery. The kernel delivers the packet to the listening socket. For an inbound TCP connection, the socket is the daemon bound to the VIP (e.g. the OPNsense webGUI on port 443, if configured to listen on the VIP).

The master’s MAC for the VIP is the physical MAC of the master, or the CARP virtual MAC if the operator has configured it. The CARP virtual MAC is computed from the VHID — 00:00:5e:00:01:XX where XX is the VHID in hex. Using the virtual MAC has the advantage that the switches do not need to update their MAC tables when failover happens — the new master’s interface MAC is the same as the old master’s. The disadvantage is that some switches (especially older firmware) have buggy MAC tables for the uncommon OUI 00:00:5e.

Read-only / Safetcpdump inbound VIP— A TCP SYN to the VIP (192.0.2.1) and the SYN-ACK from the master. The traffic is on the LAN interface. The destination IP is the VIP, the source IP is the client. The fact that the master sees the SYN and replies is the proof that the local node is master for this VHID — a backup would see the SYN pass but would not respond.

$ tcpdump -nn -i igb0 'host 192.0.2.1'

12:34:56.789012 198.51.100.50.51820 > 192.0.2.1.443: Flags [S], seq 100, win 64240, length 0
12:34:56.789234 192.0.2.1.443 > 198.51.100.50.51820: Flags [S.], seq 200, ack 101, win 64240, length 0

Illustrative output

Packet flow when the local node is backup

When the local node is backup, the kernel is not the owner of the VIP. The packet flow for the same inbound frame is:

Frame arrives on the physical interface. Same as the master.
Kernel decapsulates the frame. Same as the master.
IP routing decision. The local routing table matches the destination IP to the VIP alias on the parent interface. But the kernel sees that the local node is not the master for this VHID — the carp subsystem is in BACKUP state.
Packet is dropped at the carp layer. The kernel discards the packet before PF sees it. The packet is not delivered to the local socket. PF does not log it.
No reply. The client that sent the SYN does not receive a SYN-ACK. The client’s kernel retransmits the SYN (per the TCP retransmission algorithm). If the master is healthy, the client never sees this — the SYN reaches the master via the LAN, the master replies, and the connection establishes.

The backup’s role is to discard the packets silently. If the backup processed the packets and forwarded them, the network would have two active firewalls processing the same traffic, and the LAN would see inconsistent state. The backup is an observer, not a forwarder.

Why the backup discards silently, not with a TCP RST

A common question is why the backup does not send a TCP RST (“connection refused”) when it receives a SYN destined for the VIP. The answer is the failover contract.

If the backup sent an RST, the client would treat the connection as failed. A failover, by contrast, is supposed to be transparent — the client sees a brief pause (one or two retransmissions) and then the connection establishes against the new master. The pause is acceptable; the connection reset is not.

The way the protocol achieves this is by having the backup discard the packet at the carp layer, before PF. PF does not log the discarded packet, because PF never sees it. The operator diagnosing a missed connection cannot use the PF log to find the backup’s involvement — the only evidence is in the kernel’s CARP state (ifconfig <parent>) and in the absence of the packet from the traffic capture.

The silent discard is a feature, not a bug. It is also why the operator must verify the kernel state, not the PF log, during a CARP-related incident.

A backup that processes VIP traffic is a split-brain, not a configuration error

If the local node is backup and the firewall processes traffic destined for the VIP anyway, the LAN has two firewalls forwarding the same traffic. The result is the classic split-brain:

Some clients see one firewall’s responses; others see the other.
The MAC table on the switch alternates between the two physical MACs.
State table replication (pfsync) tries to keep both nodes in sync — but the two nodes are receiving different traffic, so the state diverges.
The next failover (the original master returning, or the secondary being demoted) produces a confusion: neither node has the full state.

The split-brain is the worst-case HA failure. Recovery is manual: shut down one of the two nodes, let the other drain its state, restart the failed node, and let the election re-resolve. The discipline is to prevent the split-brain in the first place — verify the password, verify the VHID, verify the segment on which the advertisements are reaching both nodes.

Summary

The OPNsense CARP VIP is configured under Interfaces → Virtual IPs → Add with mode CARP, the interface, the IP, the VHID, the password, the advertising frequency, and the skew.
Each GUI field maps directly to kernel state on the VIP’s parent interface. ifconfig <parent> is the source of truth; there is no carp0 device to query.
The master is the destination for VIP-bound traffic. The backup discards the traffic at the carp layer, before PF sees it.
A backup that processes VIP traffic is a split-brain — the worst-case HA failure. Verify the kernel state, not the GUI, after every change.