VM deployment — running VyOS as a guest on Proxmox, VMware, KVM

Most production VyOS routers are not bare metal. They are guests on a hypervisor that the operator’s colleagues already operate: Proxmox in a small DC, VMware vSphere in an enterprise DC, plain KVM in a lab. The behavioural differences between a VM and a physical router are small but they matter — a NIC type mismatch surfaces as a throughput problem that the routing engineer cannot explain from the router alone. This lesson covers the deployment patterns and the verification steps that catch the mismatch before the box takes traffic.

NIC type choices and what they mean

A VyOS VM exposes its NICs to the Linux kernel as ordinary Ethernet devices, but the underlying driver is virtual:

flowchart TB
  subgraph Guest[VyOS guest]
    G1[eth0<br/>virtio-net driver]
    G2[eth1<br/>e1000 driver]
    G3[eth2<br/>vmxnet3 driver]
  end
  subgraph Host[Hypervisor]
    H1[tap0 / virtio backend]
    H2[e1000 emulation]
    H3[vmxnet3 paravirtualised]
  end
  G1 --> H1
  G2 --> H2
  G3 --> H3

The choice happens once, at VM creation. Changing the NIC type later requires a guest reboot and may change the interface names if the new driver enumerates in a different order. Operators who mix NIC types on the same VM — eth0 virtio, eth1 e1000 — get a box where one link is fast and one is slow and the configuration is not the cause.

Deploying on Proxmox VE

Proxmox is the common case in small DCs and labs. The deployment is a single qm invocation against the VyOS ISO uploaded to local storage.

$ 

After the install completes, the operator edits the VM to switch the boot order from ide2 (the CD-ROM) to scsi0 (the virtual disk) and disconnects the IDE device:

qm set 100 --boot order=scsi0
qm set 100 --delete ide2

Deploying on VMware vSphere

VMware deployment uses the vmxnet3 driver and a paravirtualised SCSI controller. The simplest path is to convert the VyOS ISO into a bootable VMDK using the published scripts, or to upload the ISO to a datastore and boot from CD-ROM once.

# In the vSphere GUI: New VM → Custom → Linux → Other Linux 5.x
# - CPU: 2 vCPUs, 1 core per socket
# - Memory: 2 GiB
# - SCSI controller: VMware Paravirtual
# - Disk: 32 GiB, thin provisioned
# - NIC: 3 × VMXNET3, connected to the right port groups
# - CD/DVD: ISO from the datastore, "Connect at power on"

After install, detach the CD-ROM device and the VM boots from disk.

Deploying on bare KVM

For a Linux host running KVM directly, virt-install is the canonical tool:

virt-install \
  --name vyos-router-01 \
  --memory 2048 \
  --vcpus 2 \
  --disk size=32,format=qcow2,bus=virtio \
  --cdrom /var/lib/libvirt/images/vyos-1.5.1-iso-amd64.iso \
  --network bridge=br0,model=virtio \
  --network bridge=br1,model=virtio \
  --network bridge=br2,model=virtio \
  --os-variant generic \
  --console pty,target_type=serial \
  --graphics none \
  --boot cdrom,hd

The --console pty,target_type=serial flag connects the VM’s serial console to the host’s pty. Combined with virsh console vyos-router-01, the operator has console access without a graphics layer.

How the result is validated

The first boot into the installed image exposes the guest NICs as eth0, eth1, eth2. The operator confirms the driver with ethtool:

vyos@vyos:~$ ethtool -i eth0
driver: virtio_net
version: 1.0.0
firmware-version:
bus-info: 0000:00:03.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: no
supports-register-dump: no
supports-priv-flags: no

If ethtool -i eth0 shows driver: e1000 but the operator specified virtio, the hypervisor is misconfigured; the boot must be redone.

vyos@vyos:~$ show interface ethernet eth0
eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 52:54:00:12:34:56 brd ff:ff:ff:ff:ff:ff
    inet 192.0.2.10/24 brd 192.0.2.255 scope global eth0
       valid_lft forever preferred_lft forever

How traffic actually flows

sequenceDiagram
  participant Host as Hypervisor kernel
  participant Tap as tap0 / vhost-net
  participant Guest as VyOS eth0
  participant Peer as Adjacent peer

  Peer->>Host: Packet arrives on vmbr0
  Host->>Tap: Bridge forwards to tap0
  Tap->>Guest: Virtio descriptor ring (vhost)
  Guest->>Guest: FRR lookup + netfilter
  Guest->>Peer: Packet leaves on eth1

The bottleneck for a virtual router is rarely the guest CPU — it is the hypervisor’s NIC offload configuration. A VM running with virtio-net and LRO/GSO offloads enabled at both ends can sustain near line-rate on a 10 Gbps link. A VM running with e1000 and no offloads cannot.

How it fails

The production failure modes the engineer must recognise:

• NIC type mismatch. A box that was supposed to be virtio boots with e1000 because the VM template was copied from a legacy host. Throughput collapses; CPU per packet goes through the roof.
• Bridge firewall on the host. Proxmox’s per-VM firewall is enabled by default; on a routed VM it silently drops forwarded packets. The VM looks fine but cannot reach its peer.
• Wrong port group on VMware. A VMXNET3 NIC connected to a “VM Network” port group with no uplink produces a link that is up at the guest but unreachable at the adjacent router.
• MAC address duplication. If two VMs share a port group and the operator did not enable promiscuous mode, packets addressed to VM A’s MAC are only delivered to VM A and not to VM B.
• Thin-provisioned disk running out. A 32 GiB thin-provisioned VMDK with vyos-router-01 writing logs at 10 lines/sec grows; when the datastore fills, the VM freezes. Provision thick-eager zeroed for production routers.

Rollback

A failed VM deployment is recoverable by destroying the VM and re-creating it from the same script. The configuration is in /config/config.boot on the virtual disk; copy it off before destroying.

# On the host:
scp root@vyos-router-01:/config/config.boot /backup/
qm destroy 100
qm create 100 ... # same flags

Production discipline

Cross-course references

The Proxmox course’s XXVIII-Proxmox-Install and XXIX-Proxmox-Networking cover the host side of the deployment in detail. The Linux course’s V-Linux-NetConfig covers the underlying bridge and tap interfaces. The Observability course’s XII-Observability-HostAgents shows how to scrape the VM host’s metrics so the operator sees CPU steal time, which is the single best signal that a VM is over-subscribed.

Quiz