BGP policy: route maps, prefix lists and filtering · Proxmox VE

The EVPN lesson built a fabric. The fabric lesson gave it an underlay. Both stopped at the point where the Proxmox cluster starts talking BGP to something that is not a Proxmox node — a datacentre spine, a firewall, a provider edge — and at that point a question arrives that the earlier lessons did not need to answer:

What is this cluster telling the rest of the network about itself, and what is it willing to believe in return?

By default the answer to both is “everything it knows”. A BGP session without policy advertises every prefix in its table and accepts every prefix the peer sends. Inside a fabric of nodes you administer, that is fine. Across a boundary it is not, and the primitives Proxmox gives you for drawing that boundary are prefix lists and route maps.

The two primitives

They compose, and the composition is the thing to get straight before touching a command.

A prefix list is a named, ordered list of address-space tests. It answers “is this prefix one of the ones I mean?” and nothing else. It has no opinion about what to do with the route.

A route map is a named, ordered list of rules. Each rule matches on something — often a prefix list — and then permits or denies the route, optionally modifying its attributes on the way through.

Prefix lists are reusable. One list describing your VM address space can be referenced by the inbound map, the outbound map, and a map on a different controller. That reuse is the reason to build them separately rather than embedding CIDRs in every rule.

Prefix list entries

The fields, as the API and the documentation define them:

Field	API name	Meaning
Sequence Nr.	`seq`	“Order in which the entry is evaluated”
Action	`action`	`permit` or `deny`
Prefix	`prefix`	“Destination network in CIDR notation”
Prefix `>=`	`ge`	“Optional minimum prefix length”
Prefix `<=`	`le`	“Optional maximum prefix length”

ge and le are where most prefix-list bugs live, so they are worth stating precisely. An entry with a prefix and neither modifier matches only that exact prefix. 192.0.2.0/24 matches 192.0.2.0/24 and does not match 192.0.2.0/25 or 192.0.2.128/25.

Adding le 32 makes it match that prefix and every more-specific route inside it. Adding ge 25 le 32 makes it match the more-specifics and not the /24 itself.

Entry	Matches
`permit 192.0.2.0/24`	Exactly `192.0.2.0/24`
`permit 192.0.2.0/24 le 32`	`192.0.2.0/24` and everything inside it
`permit 192.0.2.0/24 ge 25 le 32`	Everything inside it, but not the /24
`permit 0.0.0.0/0 le 32`	Every IPv4 route
`permit 0.0.0.0/0`	Only the default route

That last pair is the one that produces a genuinely confusing incident. An operator writes 0.0.0.0/0 intending “everything”, gets a filter that permits the default route and nothing else, and watches the fabric lose all its specific routes while the default keeps working — so connectivity survives in a degraded, asymmetric form that is much harder to diagnose than a clean outage.

Configuration changebuild a prefix list describing your VM address space— Creates the list and one entry. Prefix lists on their own change nothing - they are only tests. Nothing is applied to the fabric until a route map references the list and a controller references the map, and the SDN configuration is applied.

pvesh create /cluster/sdn/prefix-lists --id pve-vm-space

pvesh create /cluster/sdn/prefix-lists/pve-vm-space/entries \
--seq 10 --action permit --prefix 192.0.2.0/24 --le 32

pvesh create /cluster/sdn/prefix-lists/pve-vm-space/entries \
--seq 20 --action permit --prefix 198.51.100.0/24 --le 32

pvesh get /cluster/sdn/prefix-lists/pve-vm-space/entries --output-format yaml

Route-map entries

Field	API name	Meaning
Order	`order`	“Between 0 and 65535. Mandatory and must be unique”
Action	`action`	`permit` or `deny`
Match	`match`	The conditions a route must satisfy
Set	`set`	Attributes to modify on a matching route
Exit Action	`exit-action`	`on-match-next`, `on-match-goto`, or `continue`
Call	`call`	“Optionally call another route map”

The match clause supports: route-type, vni, ip-address-prefix-list, ip6-address-prefix-list, ip-next-hop-prefix-list, ip6-next-hop-prefix-list, ip-next-hop-address, ip6-next-hop-address, metric, local-preference, peer, and tag.

The set clause supports: ip-next-hop, ip-next-hop-peer-address, ip-next-hop-unchanged, ip6-next-hop, ip6-next-hop-peer-address, ip6-next-hop-prefer-global, local-preference, tag, weight, metric, and src.

vni and route-type are the EVPN-specific matches, and they are what lets a policy distinguish a Type-2 MAC/IP advertisement from a Type-5 prefix route — which is the distinction that matters when you want a fabric to export its subnets but not its per-host MAC bindings.

Configuration changea route map that advertises only your VM space— Two entries. The first permits routes matching the prefix list; the second denies everything else. The order values matter and must be unique. This map is inert until a controller references it.

MAP=export-vm-space

pvesh create /cluster/sdn/route-maps/entries \
--route-map-id "$MAP" \
--order 10 \
--action permit \
--match key=ip-address-prefix-list,value=pve-vm-space

pvesh create /cluster/sdn/route-maps/entries \
--route-map-id "$MAP" \
--order 20 \
--action deny

pvesh get "/cluster/sdn/route-maps/entries/$MAP" --output-format yaml

Note the shape of the --match argument. match and set are arrays of property strings, each written as key=<name>,value=<value> — not name=value, which is the form people reach for first and which the API rejects. exit-action uses the same convention with an optional index, as key=on-match-goto,value=40.

A route map you apply and then get wrong drops every route, not some of them

Route maps carry an implicit deny at the end. A map whose entries match nothing rejects everything that was offered to it.

That is the correct and standard behaviour for a policy language, and it means the failure mode of a route map is total rather than partial. A prefix list with a typo in the CIDR does not leak a few extra routes; it matches nothing, and the map denies the entire table.

For an inbound map the symptom is that the cluster stops learning routes and loses reachability to everything outside it. For an outbound map the symptom is worse and slower: the rest of the network stops learning about your VM subnets, so traffic to your guests fails while traffic from them works — asymmetric reachability, which is the hardest kind to diagnose from either end.

Attach a policy during a window with console access, and verify with vtysh immediately, not by waiting to see whether anyone complains.

Exit actions, and the one that is easy to misread

exit-action controls what happens after an entry matches:

on-match-next — continue evaluating from the next entry.
on-match-goto — jump to a specified later entry.
continue — proceed rather than terminating on this match.

Without one, evaluation stops at the first matching entry and that entry’s permit or deny is the verdict. This is the standard first-match behaviour and it is what you want almost always.

The reason on-match-next exists is layered policy: a first entry that sets local-preference on a class of routes and then continues, so a later entry can decide whether to permit them. It is powerful and it makes a map much harder to read, which matters because the person reading it next is likely to be doing so during an incident.

call is the other composition tool — one map invoking another — and the same caution applies with more force. A two-level policy is correspondingly harder to reason about, and FRR’s own documentation is the place to go before building one.

Attaching policy to a controller

Policy that is not attached does nothing. The BGP and EVPN controllers take two properties:

route-map-in — “Route Map that should be applied for incoming routes”
route-map-out — “Route Map that should be applied for outgoing routes”

Cluster-wide riskattach the outbound policy to a BGP controller— Cluster-wide: SDN configuration is replicated and applied to every node in the controller's scope. Nothing takes effect until the SDN configuration is applied, which is the moment the fabric changes what it advertises.

CONTROLLER=bgp-spine

pvesh set "/cluster/sdn/controllers/$CONTROLLER" \
--route-map-out export-vm-space

pvesh get /cluster/sdn/dry-run --output-format yaml | head -40

The controller also carries the properties that decide what kind of BGP session this is, and they are worth reading together with the policy because a policy written for the wrong session type is a policy that does nothing:

Property	Meaning
`asn`	“autonomous system number”
`peers`	“peers address list”
`ebgp`	“Enable eBGP (remote-as external)”
`bgp-mode`	“Whether to use eBGP or iBGP. Auto mode chooses depending on BGP controller or falls back to iBGP”
`ebgp-multihop`	“Set maximum amount of hops for eBGP peers”
`loopback`	“Name of the loopback/dummy interface that provides the Router-IP”
`bgp-multipath-as-path-relax`	“Consider different AS paths of equal length for multipath computation”
`peer-group-name`	“Name of the peer group for this EVPN controller”

Verification that can fail

The SDN configuration being valid tells you FRR was given a configuration. It does not tell you what BGP is doing with it. That answer only comes from FRR.

Read-only / Safewhat are we actually advertising, and to whom— Read-only. The advertised-routes view is the one that matters for an outbound policy - it is the post-policy output, so it shows the effect rather than the intent. An empty list here after attaching a map is the implicit-deny failure.

PEER=192.0.2.1

vtysh -c 'show ip bgp summary'
vtysh -c "show ip bgp neighbors $PEER advertised-routes"
vtysh -c "show ip bgp neighbors $PEER received-routes"
vtysh -c 'show route-map'
vtysh -c 'show ip prefix-list'

Read-only / Safe— Illustrative. Two prefixes and no others is what a correct export policy looks like. Total number of prefixes 0 is the implicit-deny symptom.

$ vtysh -c 'show ip bgp neighbors 192.0.2.1 advertised-routes'

BGP table version is 14, local router ID is 10.255.0.1
 Network          Next Hop            Metric LocPrf Weight Path
*> 192.0.2.0/24     10.255.0.1               0    100  32768 i
*> 198.51.100.0/24  10.255.0.1               0    100  32768 i

Total number of prefixes 2

Illustrative output

show route-map reports a hit counter per entry, which is the fastest way to tell “my rule is wrong” from “my rule never ran”. An entry with zero hits after the session has been up for a while did not match anything, which usually means the prefix list, not the map.

Under the hood

Where SDN policy actually goes, and why vtysh is the source of truth.

Proxmox SDN is a configuration generator. Zones, VNets, controllers, prefix lists and route maps live in /etc/pve/sdn, and applying the SDN configuration renders them into the configuration files the underlying daemons read — /etc/network/interfaces.d/sdn for the Linux side, and an FRR configuration for the routing side. FRR then does the routing.

That two-layer arrangement produces the behaviour that catches people.

The SDN configuration is the intent; FRR holds the state. A prefix list in /etc/pve/sdn that has not been applied does not exist as far as BGP is concerned. A route map that was applied and then edited without re-applying is stale in one place and current in the other. This is why the verification commands in this lesson are vtysh and not pvesh — pvesh shows you what you asked for.

Policy is evaluated at the session, in a direction. An outbound map runs when a route is offered to a peer, and an inbound one when a route is received from it. Neither affects what is in the local routing table except by consequence, which is why ip route on a node is a poor place to check whether a filter worked.

Changing a policy does not reprocess established sessions automatically in every case. FRR supports route refresh, and modern peers negotiate it, so an updated policy usually takes effect without tearing the session down. When a peer does not support it, the policy applies to new advertisements and the previously exchanged routes keep their old treatment — a genuinely confusing state where the configuration and the behaviour disagree and both are internally consistent. clear bgp $PEER soft is the FRR mechanism for forcing re-evaluation, and it is worth knowing the term soft here means “without dropping the session”.

The prefix-list evaluation itself is first-match on seq, same as the route map on order, and both carry an implicit deny. The composition means a route can be dropped in two different places for two different reasons, and show ip prefix-list plus show route-map with their hit counters is how you tell which.

What happens if you attach an inbound policy and the cluster stops learning routes?

The nodes keep talking to each other and stop being able to reach anything the fabric was learning from outside — which, if the default route was learned by BGP, is everything.

The signature is specific and worth recognising, because it looks like several other faults:

BGP sessions are Established. The peers are fine.
show ip bgp neighbors PEER received-routes shows routes arriving.
show ip bgp shows few or none of them accepted.
ip route on the node has lost its non-connected routes.
Guests can reach other guests on the same VNet and nothing beyond.

The gap between received and accepted is the whole diagnosis, and it is why both commands are in the verification block above. Routes arriving and not being installed means policy, not connectivity, and it skips the entire layer-1-upward checklist that this symptom otherwise invites.

Recovery, in order:

Detach the map before you debug it. The controller property is what activates the policy, so clearing it restores the pre-policy behaviour without unpicking the map itself:

pvesh set /cluster/sdn/controllers/bgp-spine --delete route-map-in

Then apply the SDN configuration and confirm routes return.

Read the hit counters before changing anything. show route-map tells you whether the entry you suspect ever matched. Zero hits on a permit entry means the match clause is wrong — usually a prefix list that is more exact than intended, missing an le.

Check the prefix list separately. show ip prefix-list with its counters isolates the two layers. A route map that matched and a prefix list that did not are different bugs.

Re-attach during a window with console access, and verify with advertised-routes and received-routes within the window rather than afterwards.

The prevention is the same discipline as any filtering change: build the policy, verify what it would match with pvesh get /cluster/sdn/dry-run, attach it when you can watch it, and treat the first five minutes after applying as part of the change rather than as the beginning of normal operations.

When you need this, and when you do not

Route maps are the most complex thing in this part, and a good number of Proxmox estates never need them.

You do not need policy if the cluster’s BGP sessions are all internal to a fabric you administer end to end. Inside a fabric, advertising everything is the point.

You need policy at a boundary: a session to a spine that belongs to the network team, a session to a firewall, an eBGP session to a provider, or an EVPN fabric where two tenants must not learn each other’s routes.

You need it urgently in one case: a Proxmox cluster with two BGP sessions to different parts of the network can, without an outbound filter, advertise routes learned from one into the other and become a transit path. The traffic between two datacentre segments starts flowing through a hypervisor’s control-plane CPU. Everything works, slowly, and nothing points at the cause.

An outbound map that permits only your own address space and denies the rest prevents that, and it is the single most valuable policy on this page.

Common mistakes

Writing 0.0.0.0/0 when you mean “every route”. Without le 32 it matches only the default route.
A prefix list with no le on an aggregate. It matches the exact prefix and none of the more-specifics inside it.
Forgetting the implicit deny. A map that matches nothing rejects everything, and the outbound version of that is asymmetric reachability rather than a clean outage.
Attaching a policy without a console session available. An inbound filter that denies everything can remove your route to the node.
Verifying with pvesh instead of vtysh. pvesh shows intent; FRR holds the state.
Reusing an order value. It is “Mandatory and must be unique”.
Reaching for on-match-next and call early. They make a policy much harder to read, and readability is what you need at 3 AM.
No outbound filter on a cluster with two external sessions. The hypervisor becomes a transit router and nothing says so.

Key takeaways

Prefix lists test address space; route maps decide what to do. Build lists separately so they can be reused.
ge and le control prefix-length matching. A bare prefix matches exactly that prefix and nothing more specific.
Route-map entries take order (unique, 0–65535), action, match, set, exit-action and call.
match route-type and match vni are the EVPN-aware conditions.
Policy attaches to a controller with route-map-in and route-map-out, and does nothing until it is attached and applied.
Both prefix lists and route maps carry an implicit deny, so a broken policy drops everything rather than leaking a little.
Verify with vtysh: advertised-routes, received-routes, show route-map and show ip prefix-list with their hit counters.
If the cluster has two external BGP sessions, an outbound filter is what stops it becoming a transit path.

Knowledge check

Why this matters

Everything else in this part fails loudly. A bond that will not negotiate flaps. An MTU mismatch stalls a migration. A bridge with no uplink means the VM has no network. You find out, and you find out quickly.

BGP policy is not like that. It fails in ways that keep working.

An outbound filter that drops everything leaves the cluster fully functional from the inside and unreachable from the outside — and since the people using the guests are outside, the report you get is “the application is down” rather than anything about routing. A missing outbound filter turns a hypervisor into a transit router, which is invisible until somebody wonders why one node’s CPU is warm. An inbound filter that is too permissive means the cluster believes whatever the peer says, which is fine until the peer says something wrong.

None of those produce an error. They produce a network that is doing exactly what it was configured to do, which is a different thing from a network that is doing what you intended.

The habit this asks for is one command in a different tool. pvesh and the GUI show you what you configured; vtysh shows you what BGP is doing with it. advertised-routes after every outbound change and received-routes after every inbound one, plus the hit counters when they disagree with expectation. That is the whole verification loop, and it is the only thing standing between a policy you believe works and a policy you can show works.

And if you take one configuration away from this lesson rather than a habit: write the outbound filter that permits only your own address space. It is two route-map entries, it prevents the transit-path failure entirely, and it is cheap enough to do on a fabric that does not need it yet.