SDN: zones, VNets, subnets

Why this matters in production

Without SDN, every multi-tenant environment requires manual bridge creation on every node and per-VLAN configuration that does not scale beyond a handful of segments. SDN lets you define zones, VNets, and subnets once at the datacenter level and have Proxmox push the configuration to every node.

Mental model

SDN has three layers:

1. Zone — a virtually separated network area. The zone type (simple, VLAN, QinQ, VXLAN, EVPN) determines the underlying technology.
2. VNet — a virtual network inside a zone. A VNet is realised as a bridge on every node that participates in the zone.
3. Subnet — an IP range inside a VNet, with optional gateway, SNAT, and DHCP.

flowchart TB
  subgraph Z[Zone: 'corp', type VLAN]
    V1["VNet 'web' (tag 100)"]
    V2["VNet 'db' (tag 200)"]
    S1[Subnet 10.10.100.0/24]
    S2[Subnet 10.10.200.0/24]
  end
  V1 --> S1
  V2 --> S2

Zone types

Creating zones, VNets, subnets (GUI)

1. Datacenter → SDN → Zones → Create: pick the zone type, set the bridge (for VLAN zones), MTU.
2. Datacenter → SDN → VNet → Create: choose the zone, set the tag (VLAN ID for VLAN zones), alias.
3. Datacenter → SDN → Subnets → Create: choose the VNet, set CIDR, gateway, optional SNAT, DHCP.

Apply changes from Datacenter → SDN → SDN (the main SDN overview panel) — pending changes are atomic.

Pending, dry-run, rollback and the global lock

The staging model gives you four operations, and knowing all four turns an SDN change from something you hope works into something you can inspect first and undo afterwards.

pvesh get /cluster/sdn/dry-run --output-format yaml

# every object, showing both pending and running state
pvesh get /cluster/sdn/zones --pending 1 --output-format yaml
pvesh get /cluster/sdn/vnets --pending 1 --output-format yaml

pvesh create /cluster/sdn/rollback

There is also a global lock. pvesh create /cluster/sdn/lock acquires it and returns a token; every subsequent write takes that token as --lock-token, and pvesh delete /cluster/sdn/lock releases it. Its purpose is to stop two operators staging changes into the same pending configuration at the same time — which matters more than it sounds, because the apply is all-or-nothing and carries whatever anyone else staged.

The configuration itself lives in /etc/pve/sdn, which is pmxcfs — so it is replicated to every node and read-only when the cluster loses quorum. An SDN apply is therefore one of the operations that fails during a quorum incident even though the node you are on is healthy.

CLI walkthrough

Note the API root: SDN lives under /cluster/sdn, and subnets are nested under their VNet.

pvesh get /cluster/sdn/zones --output-format yaml
pvesh get /cluster/sdn/vnets --output-format yaml
pvesh get /cluster/sdn/vnets/web/subnets --output-format yaml

pvesh create /cluster/sdn/zones --zone corp --type vlan --bridge vmbr0 --mtu 1500

pvesh create /cluster/sdn/vnets --vnet web --zone corp --tag 100 --alias "Web tier"

pvesh create /cluster/sdn/vnets/web/subnets \
  --type subnet --subnet 192.0.2.0/24 --gateway 192.0.2.1 --snat 1

Applying is a PUT on /cluster/sdn itself — described in the API as “Apply sdn controller changes && reload” — not a separate apply endpoint:

pvesh set /cluster/sdn

Attaching a VM to a VNet

Once a VNet exists, attaching a VM is the same as attaching to a bridge: pick the VNet name in the VM NIC configuration.

ip -br link show | grep -E 'vnet|br-'

IPAM and DHCP

IPAM (IP Address Management) and DHCP are in tech preview as of PVE 9.2. The IPAM plugin tracks which IPs are allocated. The DHCP option requires installing dnsmasq:

apt install -y dnsmasq && systemctl disable --now dnsmasq

DHCP ranges are configured per subnet. Use with care in production — the tech-preview status means the implementation may change.

Production considerations

Common mistakes

• Creating a VXLAN zone without a routable underlay between peer IPs.
• Forgetting to apply pending changes; thinking the configuration has been pushed.
• Applying without running the dry-run. The apply carries every pending change in the cluster, including ones you did not make.
• Assuming IPAM and DHCP are just configuration. Both are documented as tech preview; the IPAM and DNS lesson covers what that costs.
• Mixing SDN-managed VNets and hand-crafted bridges on the same uplink, producing confused routing.
• Assuming IPAM/DHCP is production-grade.

Key takeaways

• SDN has three layers: zone, VNet, subnet.
• Choose the zone type by topology (Simple / VLAN / QinQ / VXLAN / EVPN).
• Apply changes atomically from the SDN overview panel — and read pvesh get /cluster/sdn/dry-run first, because the apply carries everyone’s pending changes.
• pvesh create /cluster/sdn/rollback discards the pending configuration; the global lock at /cluster/sdn/lock stops two operators staging into it at once.
• SDN configuration lives in /etc/pve/sdn, so it is replicated and becomes read-only when the cluster loses quorum.

Knowledge check