Remote change discipline — the operational standard for changes the operator cannot see

A remote change — any change to a router the operator cannot physically reach — is the highest-risk operational action. A bad change can lock the operator out, blackhole production traffic, or destabilise a redundant pair. The discipline of the remote change is what makes it safe: every step is documented, the diff is reviewed, commit-confirm is mandatory, and the verification is automated.

The four artefacts of every remote change

flowchart TB
  A[Ticket<br/>what + why] --> B[Diff<br/>set + delete]
  B --> C[Rollback plan<br/>load pre-change file<br/>OR let the confirm timer expire]
  C --> D[Verification<br/>show + ping + check log]

The change window

The change window is the time during which the change may be applied and during which the rollback plan is in effect. A change window is:

• Announced — the team knows the window is happening.
• Bounded — the window has a start and end time.
• Supervised — a second operator is available to assist or confirm.
• Reversible — the change can be rolled back within the window.

A change applied outside a window, or in a window that is not supervised, is a change applied at the operator’s personal risk.

The full workflow

1. Ticket: "CHG-4471 - add BGP neighbor 192.0.2.2 to router-core-01"
   - Requested by: networking team
   - Approver: change manager
   - Window: 2026-08-15 22:00-23:00 UTC
   - Pre-change snapshot: save /config/pre-CHG-4471.conf
   - Rollback plan: load /config/pre-CHG-4471.conf; compare; commit-confirm 15
   - Verification: show bgp ipv4 unicast summary; ping 192.0.2.2

2. Diff (the local AS is already configured as
   `set protocols bgp system-as 64512`):
   set protocols bgp neighbor 192.0.2.2 remote-as '64512'
   set protocols bgp neighbor 192.0.2.2 address-family ipv4-unicast
   delete protocols bgp neighbor 192.0.2.2 shutdown

3. Execute (in change window, with peer):
   - SSH to router-core-01
   - configure
   - save /config/pre-CHG-4471.conf
   - paste the diff
   - compare
   - commit-confirm 15
   - exit

4. Verify (outside configure mode):
   - show bgp ipv4 unicast summary
   - ping 192.0.2.2
   - show system commit | head -5

5. If verification passes:
   - configure
   - confirm
   - save
   - exit

6. If verification fails:
   - do nothing; the confirm timer expires and the router
     reverts to the configuration it had before the commit

7. Update ticket with results.

Peer review

The peer reviews the diff before the change is applied. The peer is not the operator who will apply the change — the peer is a second operator who reads the diff and confirms it matches the ticket.

The commit archive

VyOS’s change record is the commit archive, not a log file. Every successful commit writes the resulting configuration into /config/archive/ and adds a numbered entry to the list that show system commit prints, newest first. Revision 0 is the running configuration, revision 1 is what it replaced, and so on down to the retention limit.

$ show system commit | head -5

0   2026-08-15 22:15:01 by vyos via cli    CHG-4471 add neighbor 192.0.2.2
1   2026-08-11 09:02:44 by ansible via other
2   2026-08-04 17:40:12 by vyos via cli    CHG-4402 firewall rule 40
3   2026-07-29 11:18:05 by vyos via cli
4   2026-07-22 08:55:31 by vyos via cli

Read the columns rather than the exact spacing, which moves between releases: a revision number, a timestamp, the user who committed, and how the commit arrived — via cli for an interactive session, via other for a non-interactive one such as an Ansible run. The trailing text is the commit comment, and it is only there if somebody set one.

That last point is the whole reason commit comment exists. The archive records that a commit happened and who made it; it does not record why. A ticket reference in the comment is what turns the archive from a list of timestamps into a change history:

[edit]
vyos@vyos# commit comment "CHG-4471 add neighbor 192.0.2.2"

To see what a revision actually changed, compare takes revision numbers in configure mode — compare 1 0 prints the difference between the previous configuration and the running one, in the same +/- form as an ordinary candidate diff.

Post-change verification

The verification runs from the operational shell, not configure mode, and it has to answer a specific question: did the thing the ticket asked for actually happen? For this change that is one session, in one state.

$ show bgp ipv4 unicast summary

IPv4 Unicast Summary (VRF default):
BGP router identifier 10.255.0.1, local AS number 64512 vrf-id 0
BGP table version 4
RIB entries 7, using 1344 bytes of memory
Peers 1, using 20 KiB of memory

Neighbor        V         AS   MsgRcvd   MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcd
192.0.2.2       4      64512        12        10        0    0    0 00:03:45            3

Total number of neighbors 1

The load-bearing column is the last one. A number there is a prefix count, which means the session reached Established and exchanged an UPDATE. A word there — Idle, Connect, Active — is a state name, which means it did not, and Active in particular means the local end is retrying outbound and getting nowhere. An operator who reads “the session is listed” as “the session is up” will confirm a change that did not work.

Then the reachability and the record:

ping -c 5 192.0.2.2
show system commit | head -5

The ping proves the peer address is reachable independently of BGP, which separates “the neighbor statement is wrong” from “the path to the neighbor is down”. The show system commit proves the commit landed and is the newest revision — under commit-confirm that matters, because a change that has silently reverted still leaves the operator looking at a shell that appears healthy.

How the result is validated

show configuration commands | match 'neighbor 192.0.2.2'
show system commit | head -5

The first prints the running configuration as set lines and filters it to the change, which is the only form that can be compared against the diff in the ticket line for line. The second confirms the commit is the newest archive revision.

The third check runs in configure mode and answers a different question — whether the change survives a reboot:

[edit]
vyos@vyos# compare saved

compare saved diffs the candidate against /config/config.boot. Empty output means candidate, running and saved all agree. Any output at all after a confirmed change means save was skipped, and the change is already scheduled for deletion at the next reboot.

How it fails

The production failure modes the engineer must recognise:

• Change applied without peer review. A typo in the diff reaches the box; the operator notices only at runtime.
• Change applied outside the change window. The change conflicts with another concurrent change; both fail.
• Change applied without commit-confirm. A bad change locks the operator out; the box must be recovered via console.
• Change applied without verification. The operator confirms the change and moves on; the actual change is not what the diff described.
• Change not documented. A later operator tries to debug a routing issue and cannot find what changed.

Rollback

The rollback path is documented in the ticket. Inside the confirm window there is nothing to do: the timer is the safety net, and letting it expire is a legitimate — often the correct — decision.

After confirm, the change is permanent and reverting it is a new remote change. The safe path is the pre-change snapshot the ticket required, loaded back under its own confirm timer:

[edit]
vyos@vyos# load /config/pre-CHG-4471.conf
[edit]
vyos@vyos# compare
[edit]
vyos@vyos# commit-confirm 15
[edit]
vyos@vyos# exit

load replaces the candidate with the file’s contents; compare shows exactly what committing it would undo — read it before you commit, because a snapshot taken at 22:00 also reverts anything anyone else changed since. The reversion is itself a remote change with its own diff, verification and archive entry.

Production discipline

Cross-course references

The Linux course’s V-Linux-NetConfig covers the underlying configuration model. The Ansible course’s XLII-Ansible-BeyondLinux covers how to drive the remote change discipline from an Ansible playbook. The Observability course’s XII-Observability-HostAgents covers how to alert on commit events from the router’s system journal.

Quiz