max-prefix as a security control — bounding a peer, and the outage it trades for

A prefix limit is the operator’s defence against a peer that sends far more routes than the design accounts for. A peer that legitimately advertises 800 prefixes is healthy; the same peer suddenly advertising 800,000 is a leak, a compromise, or a misconfiguration, and unbounded it can exhaust the local router’s memory, stall best-path computation, and turn a single session into a router-wide outage.

The companion lesson vyos-xxviii-05-maximum-prefix covers the knob itself and how to size it. This lesson asks the security questions that sit on either side of it: where the limit belongs among the other control-plane defences, what it deliberately does not cover, and what it costs you when it fires — because on VyOS the cost is a latching outage, and an operator who deploys this control without knowing that has bought an availability incident they did not budget for.

The threat model

The prefix limit is a bound on volume received from one peer. That is a narrow claim, and being precise about it is what stops it from being treated as general-purpose protection:

• A peer with a configuration error that starts re-advertising its other upstream’s full table instead of the agreed customer cone. This is the classic route leak and it is what the control was designed for.
• A compromised or hijacked peer injecting bulk prefixes to destabilise the local table.
• An internal session — a new iBGP peer, or a router-reflector client — that turns out to be carrying the full table into a part of the AS designed for a few thousand routes.

Where it sits among the other controls

flowchart TD
  P["Peer UPDATE"] --> A["TCP transport controls<br/>MD5 / TCP-AO, TTL security, ACL"]
  A --> B["Inbound prefix-list<br/>and AS-path filter"]
  B --> C["RPKI origin validation"]
  C --> D["maximum-prefix counter"]
  D -->|under the limit| E["Loc-RIB"]
  D -->|over the limit| F["CEASE notification<br/>session down, Idle (PfxCt)"]

Read the order carefully, because it is the source of a common misconception. FRR counts prefixes as they arrive from the peer, before your inbound policy runs. A peer sending 6,000 prefixes of which your prefix-list drops 5,500 still counts as 6,000 against the limit. That is deliberate: the cost the limit bounds is the cost of receiving, parsing and evaluating the prefixes, and you pay that whether or not you keep them.

The practical consequence is that a tight filter does not let you run a tight limit. Size the limit against what the peer sends, not against what you expect to keep.

The configuration

VyOS 1.5 places the limit under the neighbour’s address family, because a prefix limit is per address family — an IPv4 limit says nothing about how many IPv6 prefixes the same peer may send:

set protocols bgp system-as 64512
set protocols bgp neighbor 192.0.2.2 remote-as 65001
set protocols bgp neighbor 192.0.2.2 address-family ipv4-unicast maximum-prefix '5000'
set protocols bgp neighbor 192.0.2.2 address-family ipv4-unicast maximum-prefix-out '2000'
set protocols bgp neighbor 2001:db8::2 remote-as 65001
set protocols bgp neighbor 2001:db8::2 address-family ipv6-unicast maximum-prefix '1000'

Two settings, two directions:

• maximum-prefix bounds what the peer may send you. This is the defensive half, and the one everybody configures.
• maximum-prefix-out bounds what you will advertise to the peer. This is the half that stops you being the incident: when an internal mistake suddenly makes 900,000 prefixes eligible for export to a customer contracted for 2,000, the export is capped rather than delivered. Upstreams increasingly require it in the peering agreement, and it is the half operators skip.

The trade this control makes

Two events, and conflating them is how operators end up surprised.

At 75% of the limit, FRR logs. The session stays up, prefixes keep being accepted, nothing else happens. 75% is FRR’s compiled-in default and, since VyOS does not expose the threshold node, it is the only value you get.

At the limit, FRR sends a CEASE notification carrying the “Maximum Number of Prefixes Reached” subcode and drops the session. The peer goes to Idle and stays there. There is no automatic retry, because the retry is the restart option VyOS does not expose.

That is the trade, stated plainly: the prefix limit converts an integrity-and-capacity risk (a peer floods your table) into a deliberate availability loss (that session is down until a human acts). For a customer session that is almost always the right trade. For your only transit session it may not be, and the decision belongs to whoever owns the availability target — not to whoever happened to be editing the BGP configuration.

Reading the down session

In show ip bgp summary, the State/PfxRcd column carries the prefix count while the session is up and the reason while it is down. This distinction is the entire diagnosis:

“The session is down and I have a prefix limit configured” is not evidence that the prefix limit dropped it. Idle (PfxCt) is.

vyos@r1:~$ show ip bgp summary

Once identified, there are exactly three decisions, and resetting the session is none of them:

1. Raise the limit, if the count that tripped it is legitimate and the number was wrong.
2. Leave it down and go and talk to the peer, if the count is not legitimate. This is the correct answer more often than operators like — the control did its job and the incident belongs to the peer.
3. Remove the limit, if it should never have been on this session at all.

Recovery, once one of those has been done, is manual:

reset bgp 192.0.2.2

The prefix count starts again from zero on the new session. Reset without fixing anything first and the count climbs back through 75%, past the limit, and the session drops again — a few minutes of table churn bought for nothing.

OSPF and IS-IS have no equivalent — say so rather than inventing one

BGP is the protocol you take routes from strangers over, and it is the one with a prefix limit. The link-state protocols have nothing comparable: neither FRR’s ospfd nor its isisd implements a prefix or LSA count limit, and the VyOS configuration tree exposes no node for one. There is no set protocols ospf max-prefix and no per-interface IS-IS equivalent; if you have seen one in a runbook, it came from another vendor’s grammar.

Confirm it on your own box rather than taking this on trust — the tree is the authority:

set protocols ospf ?
set protocols isis ?

That is not a gap you can close with a knob, and pretending otherwise is worse than admitting it. The defence for a link-state protocol is structural, and it is the area design:

• Keep external LSAs out of the areas that do not need them. set protocols ospf area 10 area-type stub bars Type-5 externals from the area entirely; adding no-summary on the ABR bars the Type-3 inter-area summaries as well. An area that cannot receive an external LSA cannot be flooded with a million of them.
• Bound what you inject. A route-map on redistribution — set protocols ospf redistribute bgp route-map RM-OSPF-OUT — is what stops a large BGP table becoming a large LSDB. Most OSPF flooding incidents are self-inflicted through an unfiltered redistribute.
• Summarise at the boundary. set protocols ospf area 10 range 10.10.0.0/16 collapses the area’s internal prefixes into one Type-3 rather than hundreds.
• Authenticate the adjacency. An unauthenticated OSPF adjacency is the mechanism by which a stranger gets to flood you at all; vyos-liii-01-routing-protocol-authentication covers it.
• Monitor the LSDB size. Since nothing will stop growth for you, show ip ospf database counts are what you alert on.

The honest summary for a design review: BGP sessions have a volume bound; OSPF and IS-IS adjacencies do not, and are kept safe by not extending them to parties you would need one for.

Failure modes

The limit is too tight — the session never stabilises

A limit sized for a customer cone applied to a full-table upstream. The session comes up, converges toward a million prefixes, trips, and latches. It looks like a flap but it is not: each cycle needs a human reset in between.

Diagnostic:

show ip bgp summary
show ip bgp neighbors 192.0.2.2

State/PfxRcd reads Idle (PfxCt), and Up/Down resets to minutes each time somebody clears it. Fix by sizing the limit from the peer’s real count with headroom — and by asking where the number came from, because it is nearly always a template written for a different kind of session.

The limit is too loose — the control is decoration

A limit of one million on a peer contracted for 2,000 prefixes. The peer leaks 850,000 routes, the local table grows by an order of magnitude, best-path computation slows, memory pressure climbs — and the limit never fires, because 850,000 is under the bound. The audit shows a prefix limit configured on every session and the incident happened anyway.

Diagnostic:

show ip route summary
show ip bgp summary

The route-summary total is far above the design figure and State/PfxRcd for that peer is a number three orders of magnitude larger than the agreement. Fix by sizing from the relationship: a customer cone gives a limit in the low thousands, a full-table transit session gives a limit in the low millions, and using the same number for both means one of them is wrong.

The limit is on the wrong address family

A generous IPv4 limit and no IPv6 limit at all — the common shape, because IPv6 is configured later and the limit is not copied across. The IPv6 table is smaller, so an IPv6 leak is a smaller number of prefixes and completely unbounded. Set a limit for every address family the peer carries.

The 75% warning goes nowhere

The mechanism warned, on time, and the warning landed in a log file nobody reads. The first human signal was the session dropping. This is the most common failure of the lot and it is not a failure on the router: the fix is in the monitoring pipeline, alerting on MAXPFX.

Somebody automated the reset

A cron job or a monitoring “self-heal” action reproduces the missing restart option. The session now flaps for as long as the peer misbehaves, and the evidence is destroyed on each cycle. Delete the job; the Idle (PfxCt) state is the point.

Rollback

A prefix-limit change is a configuration change and takes the standard path:

• compare before commit, so you see exactly what you are arming.
• commit-confirm 10 for any change made over a session that traverses the peer you are about to bound. If the limit is wrong, the router reverts it without you.
• rollback 1 then commit and save to return to the previous configuration.
• delete protocols bgp neighbor 192.0.2.2 address-family ipv4-unicast maximum-prefix then commit to remove the bound entirely. Note what this actually does: the peer is now unprotected. It is a legitimate emergency step when a session must stay up while sizing is worked out, and it must come with a note in the change record and a date to put it back.

VyOS does not check that every BGP peer has a limit. There is no validator for “unprotected peer”, and a missing limit is invisible in show ip bgp summary. The audit is show configuration commands | match maximum-prefix compared against the peer list — by hand, or by whatever configuration review you already run.

Production discipline

Cross-course references

• XXVIII-VyOS-BGPPrefixFilter (vyos-xxviii-05-maximum-prefix) covers the knob itself, the sizing procedure, and the soft-launch that replaces the warning-only mode VyOS does not have.
• vyos-liii-03-prefix-filtering and vyos-liii-04-rpki cover the controls that catch the small, targeted event a counter cannot.
• vyos-liii-01-routing-protocol-authentication covers who is allowed to form an adjacency in the first place.
• vyos-liii-05-control-plane-protection covers the TCP 179 listener and the CPU budget the limit is protecting.
• XXIII-VyOS-BGPFundamentals covers the Adj-RIBs-In / Loc-RIB distinction the counting order depends on.
• The Linux course’s XXII-Linux-NetTroubleshoot covers the table-exhaustion diagnostic on the host side.

Quiz