BGP performance — MRAI, route-refresh, soft-reconfiguration, ORF, large BGP tables

BGP performance on VyOS 1.5 LTS is governed by five mechanisms: the MRAI (Minimum Route Advertisement Interval) timer batches outbound updates so a flap storm does not flood the peer; route-refresh asks the peer to re-send its updates without resetting the session; soft-reconfiguration stores the unfiltered inbound routes locally so the operator can re-apply the inbound filter without asking the peer to re-send; ORF (Outbound Route Filtering) lets the peer apply a filter to its outbound updates so the local router receives only the routes it wants; and the BGP memory and CPU consumption grows linearly with the number of routes in the table. Each mechanism has a knob, a use case, and a production pattern.

This lesson walks the five mechanisms, the operational evidence each one leaves, and the production-grade configuration that tunes each one. The goal is the ability to choose the right knob for the right scenario — MRAI for outbound batching, route-refresh for inbound refresh, soft-reconfiguration for offline re-filtering, ORF for inbound filtering at the peer, and memory/CPU planning for large tables.

The five performance mechanisms

flowchart TD
  PERF[BGP performance] --> MRAI[MRAI timer<br/>batches outbound updates]
  PERF --> RR[Route-refresh<br/>asks peer to re-send]
  PERF --> SR[Soft-reconfiguration<br/>stores unfiltered routes locally]
  PERF --> ORF[Outbound Route Filter<br/>peer filters its outbound updates]
  PERF --> SCALE[Table scaling<br/>memory, CPU, peering]

  MRAI --> USE1[Use: flap storm protection]
  RR --> USE2[Use: inbound filter change]
  SR --> USE3[Use: offline re-filter without peer help]
  ORF --> USE4[Use: reduce inbound route count]
  SCALE --> USE5[Use: large BGP tables, IXP peering]

The diagram shows the five mechanisms and their primary use cases. The operator who understands the trade-offs can choose the right mechanism for the right scenario.

Mechanism 1 — MRAI (Minimum Route Advertisement Interval)

The MRAI timer batches outbound UPDATE messages. When the local router has multiple routes to advertise to a peer, it waits for the MRAI interval before sending the batch, and the batch carries everything that changed while it waited. Without MRAI, a flap storm would generate one UPDATE per route per flap.

One thing MRAI deliberately does not delay: RFC 4271 §9.2.1.1 exempts explicit withdrawals from the timer. A route going away is sent immediately, because delaying bad news is how you get a routing loop. Only the re-advertisement is batched, which is the asymmetry that makes a high MRAI survivable.

The knob is per-neighbour:

set protocols bgp neighbor 192.0.2.2 advertisement-interval '30'

The value is in seconds. RFC 4271 recommends 30 for eBGP and 5 for iBGP, and those are the numbers most designs are written against — but the compiled default has differed between FRR releases, so do not assume it. The router will tell you what it is actually using:

show ip bgp neighbors 192.0.2.2

prints a Minimum time between advertisement runs is N seconds line for the peer. That line, not the documentation, is the value in force.

Tuning MRAI

The trade-off:

• Higher MRAI (60s, 120s) — fewer messages, slower convergence. Use for stable peers where fast convergence is not critical.
• Lower MRAI (5s, 10s) — more messages, faster convergence. Use for critical peers where fast convergence matters.

The canonical production pattern: 30s for eBGP peers, 5s for iBGP peers. Lower values may be used for route-reflector clients where fast convergence is critical.

Mechanism 2 — Route-refresh

Route-refresh (RFC 2918) is a BGP capability that lets the local router ask the peer to re-send its UPDATE messages. The session is not reset; the peer re-sends the affected routes. The operator uses route-refresh after changing an inbound filter — instead of resetting the session (which would disrupt the production traffic), the operator asks the peer to re-send.

There is no knob to enable it — FRR advertises the route-refresh capability at session establishment. The operator triggers a refresh from operational mode:

reset bgp 192.0.2.2 soft in

The session is not dropped; the peer re-sends, and the new inbound policy is applied to what comes back.

The evidence is in the neighbour output:

show ip bgp neighbors 192.0.2.2

The Neighbor capabilities block lists Route refresh: advertised and received when both ends support it. advertised alone means you offered and the peer did not — which is the case where a soft refresh cannot work and you need the next mechanism instead.

Route-refresh vs soft-reconfiguration

The two mechanisms are similar but different:

• Route-refresh asks the peer to re-send. The peer does the work. The local router’s CPU and memory are not affected.
• Soft-reconfiguration stores the unfiltered inbound routes locally. The local router re-applies the filter when the filter changes. The peer does no work; the local router’s memory is consumed.

The trade-off:

• Route-refresh is the modern default. The peer has the routes anyway; the cost is one round-trip and a few seconds of churn.
• Soft-reconfiguration is for cases where the peer does not support route-refresh (very old peers), or where the operator wants to re-filter offline.

The canonical production pattern: rely on route-refresh. Only enable soft-reconfiguration when the peer does not support route-refresh.

Mechanism 3 — Soft-reconfiguration

Soft-reconfiguration (also called “soft-reconfig”) stores the unfiltered inbound routes locally. When the operator changes the inbound filter, BGP re-applies the filter to the stored copy without asking the peer to re-send.

The knob lives under the address family, because what gets stored is per-family:

set protocols bgp neighbor 192.0.2.2 address-family ipv4-unicast soft-reconfiguration inbound

The local router now keeps every route the peer sends, unfiltered, alongside the filtered ones. The memory cost is roughly a second copy of what that peer sends — which is why it is a per-peer decision and not a global one.

When to use soft-reconfiguration

Use soft-reconfiguration only when:

• The peer does not support route-refresh (very rare in 2026).
• The operator wants to test a new filter without affecting production traffic (the operator can apply the filter to the stored copy in a test mode).
• The operator is debugging a filter issue and needs to see what the peer sent without the filter (the stored copy is the raw inbound).

In all other cases, route-refresh is the right tool.

The evidence is indirect and worth knowing: show ip bgp neighbors 192.0.2.2 received-routes returns the unfiltered set when soft-reconfiguration is on and nothing at all when it is off. An empty received-routes on a session that is clearly carrying prefixes is not a peer problem — it is this setting being absent.

Mechanism 4 — ORF (Outbound Route Filtering)

ORF (RFC 5291) lets the local router send a filter to the peer. The peer applies the filter to its outbound updates. The local router receives only the routes that match the filter.

The canonical use case: the local router wants to receive only routes in a specific prefix range. The local router sends the prefix-list to the peer; the peer filters its outbound updates; the local router receives only the matching routes. This reduces the number of routes the local router must process.

ORF is an address-family capability, and it needs two commands rather than one — the capability plus the filter it pushes:

set policy prefix-list FROM-UPSTREAM rule 10 action 'permit'
set policy prefix-list FROM-UPSTREAM rule 10 prefix '198.51.100.0/24'
set policy prefix-list FROM-UPSTREAM rule 10 le '24'

set protocols bgp neighbor 192.0.2.2 address-family ipv4-unicast prefix-list import 'FROM-UPSTREAM'
set protocols bgp neighbor 192.0.2.2 address-family ipv4-unicast capability orf prefix-list 'send'

That second pair is the part people get wrong. ORF does not carry a separate filter of its own: what FRR pushes to the peer is the inbound prefix-list you already configured for that neighbour and family. Configuring the capability with no prefix-list import negotiates the capability and sends the peer nothing to filter on, so the full table keeps arriving and the configuration looks like it should have worked.

The three values, and what each one means from the local router’s point of view:

send is what you want for the use case in this section — reducing what arrives. receive is the favour you do a peer who is trying to reduce their own inbound load, and it costs you the CPU to apply their filter.

How ORF works

The local router sends its inbound prefix-list to the peer in an ORF message. The peer applies it to the routes it would have advertised, and sends only what passes. The local router still applies its own inbound policy to what arrives — ORF is an optimisation, not a replacement for the local filter, and a peer that ignores or mis-applies the ORF must not be able to get a prefix past you.

The evidence is in the neighbour output:

show ip bgp neighbors 192.0.2.2

The capability block reports the ORF prefix-list capability as advertised, received, or both, and the peer section reports the ORF entries in force.

The trade-off: ORF moves the filtering cost onto the peer, which is exactly why not every peer will enable it.

Mechanism 5 — Large BGP tables

A full IPv4 table from an upstream is around a million prefixes and it dominates everything else the router is doing. The two costs are memory, which scales with how many paths you keep, and CPU, which scales with how often they change.

Memory planning

The structures that consume it, and what each one scales with:

• Adj-RIBs-In — what each peer sent, per peer. Note the “per peer”: four full-table transit sessions are four copies, and inbound filtering shrinks what is kept rather than what arrives on the wire.
• Loc-RIB — the best path per prefix, once. Roughly the size of the union of the prefixes, not the sum of the peers.
• Adj-RIBs-Out — what is advertised to each peer, per peer. This is why a route reflector with fifty clients is a different machine from an edge router with two transits, even when both carry the same prefixes.

Watch the system side with show system memory, and watch it over time rather than once: the number that matters is the trend across a convergence event, not the value on a quiet afternoon.

CPU planning

The CPU cost is dominated by:

• Best-path selection. It runs per prefix whose path set changed, not across the whole table, and its cost scales with the number of candidate paths for that prefix. This is why a flap storm is expensive and a large static table is not: a million prefixes that nobody withdraws cost almost nothing to hold.
• Policy evaluation. Every route received runs the inbound route-maps and prefix-lists. A long, un-ordered route-map applied to a full-table peer is a per-prefix cost paid a million times.
• Route-flap damping. set protocols bgp parameters dampening ... tracks per-prefix history, which is a real cost of its own and the reason damping is a deliberate choice rather than a default.

Watch it from the operator shell, where the FRR daemons are ordinary processes:

ps -o pid,pcpu,rss,comm -C bgpd
top -p "$(pgrep -d, -x bgpd zebra)"

Sustained bgpd CPU during a quiet period is the signal worth alerting on; brief saturation during a convergence is expected.

Scaling strategies

For large BGP tables, the production patterns are:

• Filter inbound, aggressively. A prefix-list import that rejects what the local AS cannot use is the largest single reduction available, and it is the one under your control.
• Push the filter to the peer with ORF where the peer will accept it, so the routes are never sent.
• Take a default instead of a full table where the design allows it. Most edge routers do not need a million prefixes to make a correct forwarding decision; they need to know which of two upstreams to use, which a default plus a partial table answers.
• Dedicate the box. A route reflector with fifty clients and several million paths is a different machine from a branch router, and trying to be both is how a maintenance on one becomes an outage for the other.

How the result is validated

From operational mode:

show ip bgp summary
show ip bgp neighbors 192.0.2.2
show ip bgp neighbors 192.0.2.2 advertised-routes
show system memory

and from the operator shell, where the FRR daemons are ordinary Debian processes:

vtysh -c 'show bgp memory'
ps -o pid,pcpu,rss,comm -C bgpd

show ip bgp summary gives the peer states and the prefix count per peer, which is the first number to read when memory is the complaint. show ip bgp neighbors <peer> is where the capabilities live — route-refresh, ORF, and the Minimum time between advertisement runs line that is the MRAI actually in force. advertised-routes is the outbound view. show system memory is the box; show bgp memory is bgpd’s own accounting of it; ps is what the kernel thinks bgpd is resident in, and the three disagreeing is itself information.

The operator who watches these regularly catches performance issues before they become incidents — and, more usefully, has a baseline to compare against when one arrives.

How it fails

The production failure modes the engineer must recognise:

• MRAI too high, slow convergence. The peer has a flap; the local router does not learn about the new path for 60 seconds. The fix: lower the MRAI.
• MRAI too low, CPU saturation. The peer has many flaps; the local router’s CPU is saturated by UPDATE processing. The fix: raise the MRAI.
• Route-refresh not supported by peer. reset bgp <peer> soft in has nothing to work with. The fix: enable soft-reconfiguration inbound for that neighbour and family, and accept the memory cost as the price of not having to hard-reset the session.
• Soft-reconfiguration enabled everywhere “just in case”. The router pays a second copy of every full table it receives. The fix: enable it on the sessions you actually debug — customer and partner peers — and leave it off on full-table transits.
• ORF configured with no prefix-list import. The capability negotiates and the peer is given nothing to filter on, so the full table keeps arriving. The fix is the missing half of the configuration, not more capability.
• ORF not supported by peer. No negotiation, no error, no change in what arrives. The fix: the local prefix-list import is doing the work anyway; drop the capability so the configuration stops implying something that is not happening.
• clear ip bgp typed on 1.5. The command does not exist; VyOS 1.4 renamed the family to reset bgp. The fix is the new spelling, and the reason this belongs in a failure-mode list is that it appears in almost every BGP runbook written before 2024.
• Large BGP table, memory pressure. The kernel OOM-kills bgpd, FRR restarts it, and the re-convergence costs more memory than the steady state — so the recovery can trigger the next kill. The fix: filter inbound, drop soft-reconfiguration where it is not earning its keep, consider taking a default instead of a full table, and add memory. Aggregation is not on this list, because it changes what you advertise and not what you hold.

Rollback

Performance-tuning changes are configuration changes. The standard rollback paths:

• delete the specific node you added — delete protocols bgp neighbor 192.0.2.2 advertisement-interval, delete protocols bgp neighbor 192.0.2.2 address-family ipv4-unicast soft-reconfiguration, delete protocols bgp neighbor 192.0.2.2 address-family ipv4-unicast capability orf — and commit. A targeted delete is preferable to a whole-configuration revert on a router carrying sessions.
• reset bgp <peer> soft in to re-apply inbound policy after a filter change, and reset bgp <peer> soft out to re-advertise after an outbound one. Neither drops the session.
• load /config/archive/<known-good-file>, then compare, then commit-confirm, for a broader revert. Prefer this to rollback N, which applies the revision and then restarts the router — on a box with full tables that is a multi-minute re-convergence, not a rollback.

Two of these changes do not take effect on the existing session. Removing an ORF capability renegotiates only at session establishment, and a deterministic-med-style comparison change does not re-sort what is already held. Plan for a reset where the change is capability-level, and say so in the ticket.

Production discipline

Cross-course references

The Linux course’s XIX-Linux-NetFoundations covers the kernel TCP retransmission. The OPNsense course’s XXX-OPNsense-DynamicRouting covers the equivalent FRR BGP behaviour on the firewall side. The BGP lessons vyos-xxiv-02-bgp-neighbour, vyos-xxv-02-bgp-aggregate, and vyos-xxviii-01-prefix-list-concept cover the underlying mechanisms. The lesson vyos-xxxi-03-flapping-session covers the related flapping-session causes.

Quiz