Network security: VLANs, firewall, IDS, and intrusion detection

The Proxmox firewall is a layer, not a complete defence. Real network security combines VLAN segmentation, the PVE firewall, and an IDS that catches what the firewall misses. This lesson walks through the complete picture.

VLAN segmentation

Default network design (everything on one VLAN) is convenient and unsafe. A production cluster needs at minimum:

VLAN 10 - Management       Cluster corosync, GUI, SSH, pvesh
VLAN 20 - VM traffic      Customer VM networks
VLAN 30 - Storage         Ceph replication, NFS, iSCSI
VLAN 40 - Backup          PBS traffic (PBS sync, restore)
VLAN 99 - Out-of-band     IPMI, iLO, iDRAC (NEVER on the data network)

Each VLAN is a separate broadcast domain. A VM compromised by a network attack cannot easily reach the storage or management network.

# On each node, the network interfaces file has separate bridges
# for each VLAN-trunked role

auto vmbr0
iface vmbr0 inet static
  address 10.0.10.11/24    # Management VLAN 10
  gateway 10.0.10.1
  bridge-ports ens4f0
  bridge-vlan-aware yes
  bridge-vids 10 20 30 40

auto vmbr1
iface vmbr1 inet manual
  bridge-ports ens4f0
  bridge-vlan-aware yes
  bridge-vids 20

auto vmbr2
iface vmbr2 inet manual
  bridge-ports ens4f1
  bridge-vlan-aware yes
  bridge-vids 30

auto vmbr3
iface vmbr3 inet manual
  bridge-ports ens4f1
  bridge-vlan-aware yes
  bridge-vids 40

Each VM attached to vmbr1 is on the customer VLAN only. The PVE node’s management IP is on vmbr0. Storage traffic stays on vmbr2. The out-of- band management network is on a physically separate NIC and never bridged to data networks.

For the most paranoid deployments, use separate physical NICs for each role, with separate switches. A misconfigured VLAN trunk can’t bridge traffic between roles if the traffic never traverses the same cable.

The Proxmox firewall

PVE includes pve-firewall, a packet filter with three configuration levels:

Datacenter firewall — applies cluster-wide
Node firewall — applies to one node
VM firewall — applies to one VM’s network interfaces

Rules are evaluated in order: datacenter → node → VM → IPset → default.

Aliases and IPSets

Aliases group objects (IPs, networks, ports) so rules can reference them:

# Datacenter → Firewall → Aliases
pve-firewall alias add mgmt-net "10.0.10.0/24"
pve-firewall alias add storage-net "10.0.30.0/24"
pve-firewall alias add backup-net "10.0.40.0/24"
pve-firewall alias add dns-servers "10.0.10.5,10.0.10.6"

IPSet is faster for large lists:

# /etc/pve/firewall/cluster.fw (Datacenter → Firewall → Options)
ipset:management 10.0.10.0/24
ipset:storage 10.0.30.0/24
ipset:backup 10.0.40.0/24

# Then reference in rules
pve-firewall group add datacenter-mgmt
pve-firewall group modify datacenter-mgmt -ipset management

Rules

# Default: allow established, drop new inbound from outside
pve-firewall cluster.fw add rule --direction in --action ACCEPT \
  --source mgmt-net --dest mgmt-net --proto tcp --dport 8006 \
  --comment "Allow PVE GUI from management network"

pve-firewall cluster.fw add rule --direction in --action ACCEPT \
  --source storage-net --dest mgmt-net --proto tcp --dport 22 \
  --comment "Allow SSH from storage network"

pve-firewall cluster.fw add rule --direction in --action DROP \
  --source !mgmt-net --dest mgmt-net --proto tcp --dport 22 \
  --comment "Block SSH from non-management networks"

The --source !mgmt-net is important: it explicitly denies SSH from anywhere except the management network. A compromised VM cannot SSH to the host.

VM-level rules

Per-VM rules protect the host from a compromised VM:

# For VM 100 (a web server)
pvesh set /nodes/pve-01/qemu/100/firewall/options \
  --enable 1 \
  --input ACCEPT --output ACCEPT

pve-firewall vm.fw add rule --vmid 100 --direction in --action ACCEPT \
  --proto tcp --dport 80,443 \
  --source 0.0.0.0/0 \
  --comment "Allow HTTP/HTTPS from anywhere"

pve-firewall vm.fw add rule --vmid 100 --direction in --action DROP \
  --source mgmt-net --dest 0.0.0.0/0 \
  --comment "Block VM from initiating connections to management"

The “block VM from reaching management” rule is critical. If the VM is compromised, the attacker cannot pivot to the management network where the cluster lives.

IDS / IPS: Suricata

The PVE firewall is fast but signature-less — it only matches what you explicitly rule on. An IDS like Suricata inspects traffic for malicious patterns.

For a PVE cluster, deploy Suricata on the cluster network mirror port:

apt install -y suricata
# /etc/suricata/suricata.yaml
# af-packet:
#   - interface: eth0
#     cluster-id: 99
#     defrag: yes
#     copy-mode: ips
#     copy-iface: eth1

The copy-mode ips mirrors traffic from eth0 (the production NIC) to eth1 (the monitoring NIC), where Suricata analyzes it.

Tune Suricata rules:

# /etc/suricata/rules/local.rules
# Alert on SSH brute-force
alert ssh any any -> any 22 (msg:"SSH brute force"; \
  flow:to_server; \
  detection_filter:track by_src, count 30, seconds 60; \
  sid:1000001; rev:1;)

# Alert on Cobalt Strike C2 patterns
alert http any any -> any any (msg:"Cobalt Strike beacon"; \
  content:"net view"; nocase; \
  sid:1000002; rev:1;)

# Alert on PVE login from unusual IP
alert tls any any -> any 8006 (msg:"PVE login attempt"; \
  flow:to_server; \
  sid:1000003; rev:1;)

Run Suricata in IDS mode (alert only) or IPS mode (drop traffic on match). IDS is safer for production; IPS can drop legitimate traffic on false positives.

Responding to detected intrusions

When an IDS alerts, follow this playbook:

1. Confirm the alert is real (not a false positive)
2. Identify the affected system (which IP, which VM, which node)
3. Isolate the affected system (block outbound network)
4. Capture forensics (memory dump, log snapshot)
5. Investigate (how did they get in?)
6. Remediate (patch, restore from clean backup)
7. Document the incident and update detection rules

A scripted response:

#!/bin/bash
# /usr/local/bin/incident-isolate.sh
# Usage: incident-isolate.sh &lt;vm-id&gt;

VMID=$1
NODE=$(pvesh get /cluster/resources --type vm --vmid $VMID -o node)

echo "Isolating VM $VMID on $NODE"
# Block all VM traffic at the firewall
pve-firewall vm.fw add rule --vmid $VMID --direction out --action DROP \
  --source 0.0.0.0/0 --dest 0.0.0.0/0 --comment "Isolated by incident response"

# Snapshot for forensics
vzdump $VMID --mode stop --dumpdir /forensics --storage forensics-nfs

# Notify
slack-cli post "#incidents VM $VMID isolated for investigation"

Keep the playbook, the script, and the notification templates in version control. Practice the response quarterly.

Common mistakes

Single VLAN for everything. The most common network security mistake. Segmentation is free defence.
VM can reach management network. A compromised VM pivots to the host, then the cluster. Block VM-to-management by default.
IDS in IPS mode without false-positive tuning. Suricata’s default ruleset has hundreds of false positives that will block legitimate traffic. Run in IDS mode for the first month, then enable IPS for specific rule sets after tuning.
No incident response playbook. When an alert fires, the on-call engineer shouldn’t be Googling the procedure. Document the playbook, practice it.

Production considerations

Network tap vs SPAN port. A physical tap is more reliable than a switch SPAN port but costs more. For most clusters, a managed switch’s SPAN port is sufficient.
IDS performance. Suricata at 10 Gbps needs a modern CPU with AES-NI and a tuned rule set. For multi-Gbps clusters, consider hardware offload (Intel FM10K) or a dedicated monitoring host.
Log retention. IDS alerts are most valuable correlated with other events. Retain at least 90 days of Suricata logs.
False positive rate. An IDS that alerts 1000 times a day is useless. Tune to under 5 alerts per day per category.

Key takeaways

VLAN segmentation: management, VM, storage, backup, out-of-band.
Block VM-to-management traffic by default.
Run Suricata in IDS mode first, tune, then consider IPS.
Have a documented incident response playbook.