Virtualisation — virtio vs e1000 vs vmxnet3 on Proxmox · OPNsense

Most OPNsense firewalls in lab and small-production environments run as virtual machines on Proxmox VE. The choice of virtual NIC model — virtio, e1000, vmxnet3 — has a direct impact on throughput, CPU cost, and feature availability. This lesson covers which model to choose, how to configure multi-queue correctly, and the failure modes that surface when the wrong choice is left in production.

The three NIC models on Proxmox

Model	Emulated	Pros	Cons
virtio (paravirtualised)	None — guest driver talks directly to hypervisor	Best performance, lowest CPU cost, multi-queue, modern features	Requires the guest OS to ship a virtio driver (OPNsense and FreeBSD both do)
e1000	Intel 82574L emulated	Universally supported, no driver needed in guest	Older, capped at ~3 Gbit/s, no multi-queue, higher CPU cost
vmxnet3	VMware paravirtualised	Fast in VMware	Not paravirtualised on KVM — emulated as a different device, driver compatibility issues, no advantage over virtio on Proxmox
VirtIO (single)	virtio with single queue	Same as virtio	Capped at what one vCPU can deliver (~3–5 Gbit/s)
VirtIO (multi-queue)	virtio with N queues	Scales with vCPU count; 8 queues = up to ~25 Gbit/s	Requires guest tuning; each queue adds a vCPU cost

The safe choice on Proxmox is virtio with multi-queue enabled, with the queue count set to the number of vCPUs allocated to the firewall.

Read-only / Safeqm config NICs— The NIC configuration for VM 101. Each netX line is a virtual NIC: model (virtio), MAC, bridge to attach to, optional 802.1Q tag (for VLAN trunking — covered in Lesson 35), Proxmox firewall flag, and queues. queues=4 means the vNIC exposes 4 receive and transmit queues, one per vCPU. Without queues set, Proxmox defaults to queues=1 (single queue, no scaling).

root@pve:~# qm config 101 | grep -E 'net[0-9]+:'

net0: virtio=4A:5E:60:DD:EE:FF,bridge=vmbr0,firewall=1,queues=4
net1: virtio=4A:5E:60:DD:EE:FE,bridge=vmbr1,firewall=1,queues=4
net2: virtio=4A:5E:60:DD:EE:FD,bridge=vmbr0,tag=10,firewall=1,queues=4
net3: virtio=4A:5E:60:DD:EE:FC,bridge=vmbr1,tag=20,firewall=1,queues=4

Illustrative output

Why virtio with multi-queue is the right choice

The virtio NIC is paravirtualised: instead of emulating a hardware device, the guest driver and the hypervisor share a well-defined data path (the virtio ring buffers). Compared to e1000 emulation:

Lower CPU cost. Each packet crosses the host/guest boundary once via the virtio ring. With e1000 emulation, every register read/write traps into the hypervisor.
Higher throughput. Modern virtio implementations deliver 10+ Gbit/s on a single vNIC with multi-queue.
Multi-queue. With queues=N, the vNIC exposes N receive and N transmit queues. Each queue can be serviced by a different vCPU. RSS on the guest side distributes packets across queues → across vCPUs.

Multi-queue requires two things to work:

The vNIC is configured with queues=N in the VM configuration (e.g. queues=4).
The guest has at least N vCPUs so each queue can be serviced by a different vCPU.

If the guest has 2 vCPUs but the vNIC has 4 queues, the hypervisor will schedule 2 queues per vCPU; you get the throughput of 2 queues, not 4.

The failure modes of e1000 and vmxnet3

e1000

The Intel 82574L emulation (e1000) is solid and works everywhere. The performance ceiling is ~3 Gbit/s and there is no multi-queue. For a home or small-office firewall on a 1 Gbit/s link, e1000 is fine. For anything faster, it is the bottleneck.

The failure mode in production: the operator deploys a firewall as an emergency measure using e1000 (because it is the default in some templates) and forgets to change it. Throughput caps at ~3 Gbit/s; the operator assumes the underlying storage or CPU is the bottleneck.

vmxnet3 on KVM

vmxnet3 is VMware’s paravirtualised NIC. On Proxmox (KVM), it is emulated via a device-model — the guest sees a vmxnet3 device, but the host runs a translation layer. The performance is no better than virtio and the driver support is uneven (FreeBSD has had intermittent vmxnet3 bugs). Do not use vmxnet3 on Proxmox.

How virtio rings work

A virtio NIC has a pair of ring buffers in shared memory: one for the guest to post receive buffers (the RX ring) and one for the guest to post transmit buffers (the TX ring). The hypervisor reads from the TX ring, copies the packet out of guest memory (or uses vhost-net to do so without a guest context switch), and forwards it to the physical NIC or the opposite ring.

The data path is: guest writes a buffer descriptor to the ring, kicks the hypervisor (a small notification, often a MMIO write), the hypervisor processes the descriptor. With vhost-net, the data path stays in kernel context on the host, avoiding the user-space round trip.

For multi-queue, the rings are replicated: 4 vCPUs means 4 TX rings and 4 RX rings, each serviced by an interrupt affinitised to one vCPU. RSS on the guest side chooses which ring to use based on the packet’s 5-tuple hash.

VLAN trunking on a single vNIC

A common Proxmox design: the firewall VM has one or two vNICs attached to a Proxmox bridge, and uses 802.1Q tags to carry multiple VLANs across those vNICs. The configuration in the guest is a parent interface (the vNIC) with VLAN interfaces (children) created on top.

In OPNsense: Interfaces → Assignments → VLAN to create a VLAN interface on top of the parent vNIC. The parent vNIC is the trunk; each VLAN interface is a sub-interface on a specific 802.1Q tag.

On the Proxmox side, the bridge must carry the tagged frames to the physical NIC. By default, Proxmox bridges are VLAN-aware (Linux bridge driver with vlan_filtering=1). The VM’s vNIC receives the tagged frame intact, and the guest OS strips the tag and exposes it as a VLAN interface.

Lesson 35 covers 802.1Q trunking in detail; the relevant fact for now is that a single virtio vNIC can carry multiple VLANs.

How to verify NIC model and performance

Inside the firewall, the NIC model is exposed in pciconf:

pciconf -lv | grep -A 3 vtnet

vtnet is the FreeBSD name for virtio-net. em is e1000. vmx would be vmxnet3 (rare on FreeBSD).

For throughput verification:

netstat -I vtnet0 -w 1
vmstat 1

Sustained vtnet0 throughput on a properly configured Proxmox host with multi-queue should approach line rate on a 1 GbE link and reach 8–10 Gbit/s on a 10 GbE link.

Production patterns

Three patterns cover most cases:

Home / small office (1 Gbit/s link): virtio with queues=2, 2 vCPUs, 2 GB RAM. Single trunk vNIC carrying the WAN and LAN VLANs.
Small business (1–2.5 Gbit/s link): virtio with queues=4, 4 vCPUs, 4 GB RAM. Two trunk vNICs — one for WAN, one for LAN-side VLANs.
Datacenter (10 Gbit/s link): PCI passthrough (Lesson 36) with an Intel X710, 8 vCPUs, 8 GB RAM. Virtualisation overhead is removed entirely.

Summary

Use virtio on Proxmox for OPNsense. Avoid e1000 and vmxnet3 in production.
Configure queues=N matching vCPU count. Multi-queue is what makes virtio scale.
A single virtio vNIC can carry multiple 802.1Q-tagged VLANs as a trunk.
For 10 Gbit/s, prefer PCI passthrough over virtual NICs.