high riskservice affecting~60 min

Runbook: Modify Route Policy Safely

1 · Prerequisites

Confirm every item is in place before any state change.

2 · Pre-checks

Read-only diagnostic commands. If any of these don't match expected output, stop and investigate further.

· The full reference graph of the object being edited is established BEFORE any edit: show configuration commands | match <object-name> on this router, naming every route-map that matches on it and every neighbour, redistribution and address-family that names those route-maps. A prefix-list is a shared object; editing it changes every consumer at once.
· The intended outcome is written as a prefix-level statement that can be tested: "203.0.113.0/24 stops being advertised to the peer at 198.51.100.1, everything else is unchanged". "Tidy up the outbound policy" cannot be verified and therefore cannot be rolled back with confidence.
· A before picture captured to a file per affected neighbour: show ip bgp summary, show ip bgp neighbors <peer> advertised-routes, show ip bgp neighbors <peer> received-routes, and the route counts from each. Without a before picture, "the right routes are there" is an opinion.
· Current hit counts recorded with vtysh -c "show route-map <name>" and vtysh -c "show ip prefix-list <name>" before anything is cleared, so that a post-change counter has something to be compared against.
· Whether the route-map ends in a catch-all permit established by reading it, not by assuming. A route-map falls off the end into an implicit deny, so adding a rule to a route-map that has no catch-all can filter routes that were previously passing.
· Whether any set community in the change carries additive confirmed line by line. Without it the set clause REPLACES the community set, discarding whatever the upstream attached — including well-known communities such as no-export.
· The peer's maximum-prefix limit known, on both sides, if the change can increase what this router advertises or accepts. A policy edit that leaks a few thousand extra prefixes can trip the far end into shutting the session down.
· Out-of-band console confirmed. Route policy governs the paths this router advertises and accepts, and that routinely includes the path your SSH session is riding.
· Change window, ticket and a named peer reviewer for the diff. The reviewer reads compare output, not a description of the intent — the two differ more often than anyone expects.
· A soak plan agreed: how long the change is watched before confirm, and who is watching what. Route policy failures are frequently silent at the router and loud at a customer three hops away.

3 · Procedure

Execute each step in order. Verify the expected output of a step before moving to the next.

1Map the reference graph: show configuration commands | match <object-name>, then show configuration commands | match <route-map-name> for every route-map that surfaced. Write the list of affected neighbours and directions into the ticket. This is the blast radius, and it is almost always wider than the object being edited.
2Capture the before picture to files: per affected neighbour, show ip bgp neighbors <peer> advertised-routes and show ip bgp neighbors <peer> received-routes, plus show ip bgp summary and show ip route summary. Record the counts, not just the output.
3Record current hit counts: vtysh -c "show route-map <name>" and vtysh -c "show ip prefix-list <name>". These are the only evidence that will show whether your new rule fires at all.
4Read the whole route-map before editing it, in sequence order, and identify the first rule that will match the prefixes you care about. First match wins and later rules are never consulted, so a broad permit at a low sequence number silently defeats a specific deny added later.
5Prefer a parallel object to an in-place edit where the topology allows it: build a new route-map at a new name, verify it exists and renders, then switch the neighbour binding in a single commit. The revert is then one line rather than a reconstruction of the previous rules.
6Where an in-place edit is unavoidable, add rules at new sequence numbers with gaps left between them. VyOS has no renumber operation — a rule at the wrong sequence must be deleted and re-created, so choosing the number correctly the first time is cheaper than fixing it under pressure.
7Check the tail of the route-map: if the intent is "filter these, pass everything else", there must be an explicit catch-all permit at the highest sequence. Without it the implicit deny filters everything that falls through, which on a redistribution turns the change into a blackhole.
8Run compare in configure mode and read every line of the output, including the lines you did not expect. Have the reviewer read it too. The diff is the change; the intent is not.
9Apply with commit-confirm <minutes> sized to the WHOLE verification window — the refresh, the convergence and the validation below — not the reflexive five. Then leave configure mode to run the validation.
10Make the change take effect: clear ip bgp <peer> soft in for an inbound policy change, clear ip bgp <peer> soft out for an outbound one. A committed policy does not retroactively re-evaluate routes already in the table, so until the refresh the router looks unchanged and the validation below is measuring the old state.
11Walk the validation ladder in order: the VyOS tree (show policy route-map, show policy prefix-list), the FRR view (vtysh -c "show route-map <name>", vtysh -c "show ip prefix-list <name>"), the hit counts, and finally the routes themselves. A disagreement between any two adjacent rungs names the fault.
12Test the matcher directly rather than inferring it: vtysh -c "show ip prefix-list <name> <prefix>" returns permit or deny for one prefix against one list. Run it for a prefix the rule must match AND for a prefix it must not.
13Diff the after picture against the before picture per neighbour: show ip bgp neighbors <peer> advertised-routes and received-routes again, and account for EVERY prefix that appeared or disappeared. An unexplained delta is the change doing something nobody asked for.
14Confirm the sessions survived: show ip bgp summary for every affected neighbour, checking that no session reset and no peer tripped its maximum-prefix limit during the refresh.
15Soak for the agreed window with the downstream watchers reporting, then confirm and save. Saving is a separate act from confirming; a confirmed change that was never saved disappears at the next reboot.
16Close the ticket with the diff, the before and after counts per neighbour, and the hit counts. The next person to edit this policy needs to know what it looked like before you touched it.

4 · Verification

Confirm the procedure actually fixed the problem.

✓compare output before the commit matched the intended change exactly, with no incidental lines, and was read by a second person
✓show policy route-map <name> and vtysh -c "show route-map <name>" agree. A disagreement means the commit did not fully render and the running policy is not the configured one.
✓vtysh -c "show ip prefix-list <name> <prefix>" returns the intended verdict for a prefix the change must affect, and the unchanged verdict for a control prefix it must not
✓The new rule has a non-zero hit count in vtysh -c "show route-map <name>" after the soft refresh. A zero hit count on a rule that should be matching means the sequence ordering or the match clause is wrong, regardless of how correct the configuration looks.
✓show ip bgp neighbors <peer> advertised-routes differs from the before picture in exactly the prefixes the change intended, and the count delta is fully accounted for
✓show ip bgp neighbors <peer> received-routes for an inbound change likewise, with every appearing or disappearing prefix explained
✓show ip bgp summary shows every affected session still Established with an uptime spanning the whole change — no session reset during the refresh, and no peer shut down by maximum-prefix
✓show ip route summary is within the expected range for this router; a table that lost thousands of routes during a policy edit is an implicit deny that nobody meant to write
✓A real destination behind an affected prefix is reachable from a real source, tested after the refresh rather than before it
✓The change is confirmed AND saved: show commit-confirm reports no pending timer, and the configuration survives a show system commit review

5 · Rollback

If verification fails, undo the procedure in reverse order.

↶Nothing before the commit needs rolling back. compare and discard in configure mode abandon a candidate that was never applied, and the running policy is untouched.
↶Inside the commit-confirm window the cheapest revert is to do nothing and let the timer fire, or to type cancel for an immediate rollback. Both restore the previous configuration without you having to reconstruct it.
↶After confirm, revert with rollback 1, then compare, then commit, then save. Read the compare output — rollback reverts the entire tree to that archive point, including anything a colleague committed in the same window.
↶A parallel-object change reverts by pointing the neighbour binding back at the previous route-map in one line, which is why it is worth the extra setup on a high-risk edit.
↶ROLLING BACK THE POLICY DOES NOT ROLL BACK THE ROUTES. The BGP table holds what the old policy produced until it is re-evaluated, so the revert needs its own clear ip bgp <peer> soft in or soft out exactly as the change did. A revert that stops at commit leaves the router configured one way and forwarding another.
↶A session that a peer shut down on maximum-prefix does not return because the policy was reverted. It comes back on its own only if maximum-prefix-restart is configured; otherwise it needs a clear ip bgp <peer>, which is a hard reset with its own route-burst cost.
↶A set community that replaced rather than added has already destroyed attributes on routes this router re-advertised. Reverting the policy stops the damage but the downstream copies persist until every affected neighbour has re-received the prefixes.
↶Deleting a whole policy object that other route-maps still reference is not a rollback, it is a second outage. Check the reference graph again before any delete policy — the same graph you built in step one.
↶If the rollback itself is a remote change on the affected path, it gets commit-confirm too. The recovery is a change and it inherits every risk the original had.

6 · Escalation

When the runbook isn't enough, contact:

· Escalate to network engineering before editing any policy object referenced by more than one neighbour or address-family. A shared object edited during an incident is how one peer's fix becomes four peers' outage.
· Escalate to the peer's owner if the session drops or trips maximum-prefix during the refresh. The far side may have shut the session down administratively, and no local change brings it back.
· Escalate to whoever owns the estate's community policy before changing any set community clause. Communities are a contract with upstreams and downstreams, and stripping one silently changes how other people's routers treat your routes.
· Escalate to security if the policy in question implements a filter that exists for security reasons — a bogon filter, an anti-leak prefix-list, an RPKI-adjacent policy. Loosening it is a security decision, not a routing one.
· Escalate immediately if the after picture shows prefixes this AS should not be advertising. That is a route leak in progress and it has its own runbook and its own clock.
· Escalate to the change owner when the diff is bigger than the intent. A compare that shows lines you did not type means somebody else is editing the same candidate, and two operators sharing one configuration session is a problem to solve before committing anything.
· Hold rather than force: a policy that is imperfect but stable, with a named owner and an agreed window, is a far better position than a rushed edit to a shared object outside a change window.

A route policy edit is not like other router changes, and the difference is worth stating before any command.

The object is shared. A prefix-list is not owned by the route-map you are editing it for. It may be matched by four route-maps, which may be bound to six neighbours in two directions. Adding one line to it changes all of them simultaneously, and nothing in the edit itself tells you so.

The failure mode is silence. A route-map falls off its end into an implicit deny. The commit succeeds, the configuration is syntactically perfect, and the prefixes that used to fall through the bottom of the route-map are now filtered. Nothing errors. The router does not complain. Somebody three hops away notices.

The commit is not the change. BGP does not retroactively re-evaluate routes it has already accepted or advertised when the policy changes. Until the session is refreshed, the router runs the new policy against nothing, and every validation command you run is describing the old state. This is the single most common way a policy change is declared successful and then surprises everyone twenty minutes later, when something unrelated triggers the refresh.

Everything below is built around those three facts.

When to use this runbook

Adding, removing or re-ordering rules in a route-map that is bound to a live neighbour, redistribution or address-family.
Editing a prefix-list, community-list or as-path-list that a live route-map matches on.
Changing a set clause — metric, local-preference, community, AS-path prepend — on any policy in production.
Attaching an existing policy to a neighbour or direction that did not have one.

When not to use it

Policy-based routing. policy route-map bound to an interface is a forwarding-path mechanism with different failure modes and a different runbook. This one is about what the router advertises and accepts.
A prefix that is missing right now. Diagnose first. A policy edit made to chase a symptom you have not localised is how one missing prefix becomes twelve.
An emergency leak stop. If this AS is currently advertising something it must not, that has its own runbook and its own urgency; the deliberate pace here is wrong for it.
First-time policy design. Composing a policy from scratch belongs in a design review with a lab behind it, not in a change window.

Blast radius

Every prefix the policy touches, on every neighbour that references it, in the direction it is bound — and, because policy objects are shared, frequently on neighbours nobody was thinking about.

The refresh itself has a cost. clear ip bgp <peer> soft in asks the peer to re-send under the route-refresh capability, or re-evaluates a locally stored copy where soft-reconfiguration is enabled; the session stays up and installed routes are only disturbed where the new policy actually changes the outcome. That is cheap. A bare clear ip bgp <peer> is not: it resets the session, withdraws every prefix from that peer, and then takes a full-table burst back.

Inputs

The object being edited and the exact intended outcome, at prefix level.
The reference graph: every route-map, neighbour, direction and redistribution that reaches this object.
The before picture, captured to files.
The peer’s maximum-prefix limits on both sides.
The reviewer, the change window, the soak length and the watchers.

Step 1: Find everything that references the object

Read-only / Safethe blast radius is a grep, not a guess

show configuration commands | match PL-CUSTOMER-OUT
show configuration commands | match RM-PEER-A-OUT
show configuration commands | match route-map
show policy route-map
show policy prefix-list

Start from the object you are about to edit and work outwards. The prefix-list names the route-maps that match on it; those route-maps name the neighbours, address-families and redistributions that use them. Write the resulting list into the ticket. That list — not the object — is what you are changing.

Step 2: Capture the before picture

Read-only / Safethe only thing 'it looks right' can be compared against

show ip bgp summary
show ip route summary
show ip bgp neighbors 198.51.100.2 advertised-routes
show ip bgp neighbors 198.51.100.2 received-routes

Save the output per affected neighbour, and record the counts as numbers. After the change you are going to subtract, and “roughly the same amount” is not a verification. Every prefix that appears or disappears has to be explained by the intent — the unexplained ones are the change misfiring.

Record the hit counts too, before anything gets cleared:

Read-only / Safehit counts, before

vtysh -c 'show route-map RM-PEER-A-OUT'
vtysh -c 'show ip prefix-list PL-CUSTOMER-OUT'

Step 3: Make the revert one line

Where the topology allows it, do not edit the live object at all. Build a new one, verify it, and switch the binding in a single commit.

The advantage is entirely in the rollback. Reverting an in-place edit means reconstructing rules under pressure from memory or from an archive; reverting a binding switch means pointing the neighbour back at the policy that was working ten minutes ago.

Where an in-place edit is unavoidable, two mechanics matter:

Sequence numbers decide behaviour, and VyOS cannot renumber. A rule written at the wrong sequence has to be deleted and re-created. Leave gaps — 10, 20, 30 — so the next person can insert without a rewrite.
First match wins and later rules are never consulted. A broad permit at sequence 10 defeats a specific deny added at 20, and the configuration looks entirely correct while doing the opposite of what it says. Read the route-map in sequence order and find the first rule that will match your prefixes before writing anything.

Step 4: The three clauses that bite

The third is subtler: match ip address prefix-list matches the prefix and its length. A /24 entry does not match a /16 route. A rule that looks like it covers a range covers exactly one length unless the prefix-list uses ge or le to widen it — and the symptom is a rule with a stubbornly zero hit count while the operator re-reads a match clause that is, in itself, correct.

Step 5: Read the diff, then commit against a clock

Configuration changethe diff is the change; the intent is not

compare
commit-confirm 20
exit

Have the reviewer read compare output rather than a description. Two things routinely appear there that nobody typed: a leftover from an earlier session in the same candidate, and a second operator’s edits if you are sharing the configuration.

Size the commit-confirm window to the whole verification — the refresh, the convergence, the ladder in Step 7 and the before/after diff in Step 8. Twenty minutes that you confirm early costs nothing. Five minutes that expires in the middle of validation rolls the change back under you and turns a controlled change into a surprise.

show commit-confirm shows the running timer if you lose track of it.

Step 6: The commit is not the change

Service impact possiblemake the policy take effect

clear ip bgp 198.51.100.2 soft in
clear ip bgp 198.51.100.2 soft out

Use the direction the change is bound to: soft in for an inbound policy, soft out for an outbound one. The soft form relies on the route-refresh capability, which every BGP implementation in current production supports, or on a locally stored unfiltered copy where soft-reconfiguration inbound is enabled. Either way the TCP session is not reset and no prefix is withdrawn just to re-apply a filter.

Step 7: Walk the validation ladder

Read-only / Safetree, engine, counters, routes

show policy route-map RM-PEER-A-OUT
vtysh -c 'show route-map RM-PEER-A-OUT'
vtysh -c 'show ip prefix-list PL-CUSTOMER-OUT'
vtysh -c 'show ip bgp route-map RM-PEER-A-OUT'

Each rung answers a different question, and a disagreement between two adjacent rungs names the fault precisely:

Tree vs engine. If show policy route-map and vtysh -c 'show route-map' disagree, the commit did not fully render. The router is running something other than what is configured, and no amount of route inspection will explain why.
Engine vs counters. If the rule is loaded and its hit count is zero after a refresh, it is not matching. Sequence ordering and match-clause semantics are the two causes, in that order of likelihood.
Counters vs routes. If the rule is firing and the routes are still wrong, the rule fires and does the wrong thing — usually a set clause that is not what was intended, or an earlier rule that already decided the disposition.

Then test the matcher directly rather than inferring it from route output:

Read-only / Safeone prefix, one list, one verdict— Illustrative shape, not a captured session.

vyos@vyos:~$ vtysh -c 'show ip prefix-list PL-CUSTOMER-OUT 203.0.113.0/24'

   seq 20 deny 203.0.113.0/24

Illustrative output

Run it twice: once for a prefix the change must affect, once for a control prefix it must not. The second test is what proves the edit is targeted rather than merely present.

Step 8: Diff against the before picture

Read-only / Safeaccount for every prefix that moved

show ip bgp neighbors 198.51.100.2 advertised-routes
show ip bgp neighbors 198.51.100.2 received-routes
show ip bgp summary
show ip route summary

Subtract from the before picture. Every prefix that appeared or disappeared must be explained by the intent written in the pre-checks. An unexplained delta of one prefix is a bug in the change; an unexplained delta of thousands is an implicit deny or a shared object doing what shared objects do.

Check the sessions survived while you are here. An uptime in show ip bgp summary that is shorter than the change means the session reset during the refresh, which is worth understanding before you confirm anything. A session that is down entirely may have been shut by the far end’s maximum-prefix limit — and that does not clear itself unless a restart timer is configured.

Step 9: Soak, confirm, save

Watch for the agreed window with the downstream watchers reporting, then:

Configuration changetwo separate acts

configure
confirm
save
exit

confirm cancels the rollback timer and keeps the change in the running configuration. save writes it to the boot configuration. A change that was confirmed and never saved is live right now and gone at the next reboot — which is the worst kind of latent fault, because it appears weeks later during an unrelated maintenance and nobody connects the two.

Rollback

What you did	How to undo it
Edited the candidate, no commit	`discard` in configure mode; nothing was applied
Committed with `commit-confirm`, still in window	Do nothing and let the timer fire, or `cancel` for an immediate revert
Confirmed the change	`rollback 1`, `compare`, `commit`, `save`
Switched a binding to a parallel policy	Point the binding back; one line, one commit
Any of the above, on a live session	Follow it with `clear ip bgp <peer> soft in` or `soft out` — the routes do not revert on their own
Session shut by `maximum-prefix`	Returns on its own only with `maximum-prefix-restart`; otherwise `clear ip bgp <peer>`, which is a hard reset
Replaced a community set by mistake	Revert and refresh, then verify downstream — the stripped attributes are already out there

The row that gets forgotten is the refresh. Reverting the configuration and stopping there leaves the router configured one way and forwarding another, which is harder to diagnose than the original problem because every configuration view now looks correct.

Common patterns

Observation	Likely cause	Next step
Commit succeeded, nothing changed at all	No refresh yet; the table still holds the old evaluation	`clear ip bgp <peer> soft in` or `soft out`
New rule shows zero hits after the refresh	An earlier rule already matched, or the match clause is wrong	Read the route-map in sequence order; test with `show ip prefix-list <name> <prefix>`
Far more routes disappeared than intended	Implicit deny; no catch-all permit at the tail	Add the catch-all, or revert and rewrite
Rule matches exact prefixes only	`prefix-list` matches prefix AND length	Use `ge` / `le` to widen the range deliberately
Downstream lost communities it relied on	`set community` without `additive`	Revert, refresh, then verify at the neighbour
Neighbours nobody mentioned are affected	A shared policy object	Rebuild the reference graph; consider a per-neighbour object
Session dropped during the refresh	Hard `clear` instead of `soft`, or the peer tripped maximum-prefix	Check `show ip bgp summary` uptime and the peer’s limit
Change works, disappears after a reboot	Confirmed but never saved	`save`, and add it to the change checklist

Escalation

Escalate when:

The object is referenced by more than one neighbour or address-family and the change is not in a planned window.
A session drops or trips maximum-prefix during the refresh.
The change touches communities, which are a contract with other networks.
The policy exists for a security reason and the change loosens it.
The after picture contains prefixes this AS should not advertise. Stop; that is a leak.
compare shows lines you did not type.

Why this matters

Route policy is the part of a router where a small, correct-looking edit has an unbounded radius. The configuration language does nothing to warn you: a prefix-list does not list its consumers, a route-map does not announce its implicit deny, and set community does not mention that it replaces rather than adds.

So the discipline has to supply what the syntax does not. Build the reference graph before the edit, because the object you are changing is rarely the thing you are changing. Capture a before picture, because “looks right” is not a comparison. Make the revert one line, because you will want it under pressure. And refresh deliberately, because a policy change that has not been re-evaluated has not happened yet — it is merely scheduled, for a moment you will not choose and probably will not be watching.