Skip to main content
RunBook Academy

← All runbooks in VyOS

high riskservice affecting~60 min

Runbook: Restore a VyOS Configuration

1 · Prerequisites

Confirm every item is in place before any state change.

2 · Pre-checks

Read-only diagnostic commands. If any of these don't match expected output, stop and investigate further.

  • · The question "is this a restore or a rollback?" has been answered. If the state you want is an entry in show configuration commit, this is the wrong runbook - rollback N needs no file, no transfer and no source-selection decision
  • · The source file is named exactly, with its age: an archive entry, /config/backups/<label>.boot, the off-box rsync copy, or a specific commit in the configuration repository. "The backup" is not a source; each of those has a different lag and restoring from the wrong one discards different work
  • · The gap between the source and now is understood and accepted by the change owner. Everything committed after the snapshot is about to be discarded, including changes made by people who are not in this incident
  • · Out-of-band access is confirmed and tested. A restored configuration replaces the interface addressing, the SSH binding and the firewall in one commit, so it can remove the session running it
  • · Whether the target hardware matches the hardware the file was written for is established: same interface names, same hw-id values, same addressing. A configuration is not portable between appliances by default
  • · The current configuration has been captured off-box first, even though it is the broken one. load overwrites the candidate, and the broken configuration is the only record of what went wrong
  • · A pre-restore snapshot exists on the box: save /config/backups/pre-restore.boot. The archive only records committed states, so an uncommitted or partially repaired candidate has no other way back
  • · For a restore that touches the routing plane, the peers or the change owner know it is happening. A restore re-establishes adjacencies from scratch and the reconvergence is visible from other people routers

3 · Procedure

Execute each step in order. Verify the expected output of a step before moving to the next.

  1. 1Decide whether this is a restore at all. show configuration commit lists every archived commit with its timestamp, user and source address; if the good state is in that list, stop here and use rollback N instead.
  2. 2Choose the source deliberately and write down its age. The local archive is as fresh as the last commit, /config/backups/ as fresh as the last manual snapshot, the off-box rsync copy as fresh as its interval, and the off-site archive as fresh as its daily run.
  3. 3Capture the state you are about to destroy: show configuration commands written off-box, plus the operational view - show interfaces, show ip route, show ip bgp summary, show ip ospf neighbor - so the post-restore comparison has a baseline.
  4. 4Take a pre-restore snapshot on the box with save /config/backups/pre-restore.boot, so a restore that turns out to be wrong has somewhere to go back to that is not the archive.
  5. 5Get the file onto the router by the path most likely to work. load scp://... makes the ROUTER the client and needs its DNS, routing and host keys intact; during an outage those are candidates for the thing that is broken. Pushing the file with scp from the workstation, or pasting it over the console, does not.
  6. 6Load it into the candidate: configure, then load /path/to/file. This REPLACES the candidate; it does not merge. The active configuration is untouched until commit.
  7. 7Read the whole compare output, not the part you expected. On a restore the diff is every change made since the snapshot was taken, which includes other people work, and it is the only place that becomes visible.
  8. 8If the hardware differs from the source, fix the hardware-bound nodes in the candidate BEFORE committing: interface names, hw-id values, and any address that has moved. Committing first and fixing afterwards means the box comes up with no working interface.
  9. 9Commit with commit-confirm whenever the restore includes the interface, SSH, VRF or firewall configuration that carries your session - which for a full restore is always. Then verify from OUTSIDE configure mode, and confirm only after the check passes.
  10. 10save with no argument. save /config/backups/... writes a snapshot and leaves /config/config.boot untouched, so a restore that ends that way is one reboot away from reverting.
  11. 11Verify at the protocol layer, in dependency order: interfaces up, then addresses, then adjacencies, then routes, then traffic. A configuration that matches the file proves the load worked, not that the router does.
  12. 12Reconcile the sources of truth: push the restored configuration back to the off-box copy and the repository, or the next scheduled backup captures the restored state as if it were a change nobody made.

4 · Verification

Confirm the procedure actually fixed the problem.

  • compare was read in full before the commit, and every line in it was either expected or explicitly accepted as lost work
  • show interfaces lists every interface the configuration expects, with the expected addresses and admin state - not merely the same count
  • No interface is missing because its hw-id does not exist on this hardware: show configuration commands | match "hw-id" matches what the box actually has
  • Routing adjacencies are re-established rather than merely configured: show ip ospf neighbor shows Full, show ip bgp summary shows Established with a plausible prefix count, and any BFD sessions are up
  • show ip route contains the expected routes, including the default, and the FIB agrees with the protocol tables
  • Traffic passes end to end through the router, tested from a client rather than from the router itself
  • The restore persisted: ls -la /config/config.boot shows a current timestamp and diff /config/config.boot <(show configuration commands) returns nothing
  • A new archive entry exists for the restore commit in show configuration commit, attributed to the operator who ran it
  • The off-box backup and the configuration repository have been updated with the restored state, so the next scheduled run does not report it as unexplained drift

5 · Rollback

If verification fails, undo the procedure in reverse order.

  • Before the commit, discard throws the loaded candidate away and leaves the running configuration untouched. This is the free exit and it is available right up to the moment you commit
  • While the commit-confirm timer runs, doing nothing restores the pre-restore state automatically. This is the intended path when the verification fails, and it is why a restore of the access path is committed no other way
  • After a confirmed restore, the way back is the pre-restore snapshot: load /config/backups/pre-restore.boot, compare, commit, save. This is why Step 4 exists
  • rollback 1 also works after a confirmed restore, because the restore itself wrote an archive entry - but read compare first. During an incident the previous entry may be the broken configuration you were escaping
  • POINT OF NO RETURN: work committed after the snapshot was taken. A restore discards it silently; nothing in the output names what was lost, which is why the compare output is the record
  • POINT OF NO RETURN: an archive entry that has been pruned. commit-revisions defaults to 20 and pruning is irreversible, so an older state exists only in the off-box copy or nowhere
  • POINT OF NO RETURN: a restore performed as part of a credential incident. Every pre-incident file contains the compromised secrets, so the restore must be followed by re-applying every rotation
  • A reboot is not a rollback. If save has already run, the box comes back on the restored configuration - the pre-restore state is only in the snapshot file and the archive

6 · Escalation

When the runbook isn't enough, contact:

  • · Escalate to the change owner before discarding work: the compare output names people whose changes are about to be reverted, and that is their decision as much as yours
  • · Escalate to network engineering before restoring a router in a redundant pair. A restore drops and re-establishes every adjacency, so the pair is briefly running on one member
  • · Escalate to the platform team if no source is fresh enough to be useful. A backup that predates a major change is a decision about how much to rebuild by hand, not a restore
  • · Escalate if the target hardware differs from the source: different interface names, a different port count, or different addressing turn a restore into a migration with its own review
  • · Escalate to security if the reason for the restore is suspected compromise. Restoring from a pre-incident file reintroduces whatever was disclosed, and that sequencing is not an on-call decision
  • · Escalate if the load fails on a validator. A configuration written by a different VyOS release train can be rejected by the current schema, and the fix is a migration rather than a retry
  • · Escalate if the router cannot reach the backup source and the console is the only remaining path - a large configuration pasted over a serial line is slow enough to need its own window

A restore feels like an undo and is not one. rollback N is an undo: it moves the router back along its own history, one archived commit at a time, and the box has never been anything other than itself. A restore takes a file written at some other moment, possibly for some other appliance, and replaces the entire candidate configuration with it. Everything between the file and now disappears in one commit, and nothing in the output tells you what.

That is the whole difficulty. load is a single command that always succeeds if the file parses, and the three decisions that determine whether the restore helps or hurts all happen before it: whether this should be a rollback instead, which source you picked and therefore how much recent work you are discarding, and whether the hardware under the file is the hardware the file was written for.

The commands here are short. The reading is long, and the compare output is where the work is.

When to use this runbook

  • The saved configuration is lost or corrupted and the router is otherwise running.
  • A replacement appliance needs the failed one configuration on it.
  • A change went wrong and the state you need is older than the archive holds - because it was pruned, or because the box was reinstalled.
  • A disaster-recovery drill is exercising the restore path deliberately.

When not to use it

  • The good state is an entry in show configuration commit. Use vyos-rb-rollback-bad-config: no file, no transfer, no staleness decision, and the diff is small enough to read properly.
  • You are taking the backup rather than using one - that is vyos-rb-backup-vyos-config.
  • The router is dead and this is a full rebuild including installation and hardware. The restore is one stage of vyos-rb-recover-failed-router, and the stages before it decide whether you are restoring at all.
  • The reason for the restore is a credential compromise. Restoring reintroduces every secret in the file: run vyos-rb-respond-compromised-credentials first, and treat the restore as a step inside it.

Blast radius

A full restore is the largest single change a VyOS router accepts. It replaces interfaces, addressing, routing protocols, firewall, NAT and the management plane in one commit. Every adjacency drops and re-forms. Every stateful firewall connection is re-evaluated against a rule set the kernel has just been handed. If the router is in a redundant pair, the pair is running on one member for the duration.

It also, and less obviously, discards work. The blast radius includes every change committed after the snapshot was taken, by anyone, for any reason.

Step 0: is this a restore or a rollback?

Read-only / Safecheck the archive before reaching for a file
show configuration commit
show system config-management

show configuration commit lists each archived commit with its number, timestamp, user and source address. If the state you want appears in that list, this runbook is the expensive way to get it.

SituationRight tool
Good state is a recent archive entryrollback N - vyos-rb-rollback-bad-config
Good state predates the retained archiveRestore from a file. This runbook
Archive is intact but /config/config.boot is corruptRestore from the archive entry, then save
Box was reinstalled - archive is emptyRestore from the off-box copy or the repository
Different appliance entirelyRestore, plus hardware adaptation. Step 5

The archive default retention is 20 entries. On a busy router that can be a single afternoon, which is exactly why a restore is sometimes the only option for a state that feels recent.

Step 1: choose the source, and name its lag

Every backup location trades freshness for independence, and the restore inherits whichever trade you pick.

SourceFresh as ofWhat it costs you
/config/archive/ entryThe last commitNothing, but it dies with the disk
/config/backups/<label>.bootWhenever someone last ran save <path>Manual, so often much older than it looks
Off-box copy from the scheduled syncIts intervalUp to one interval of committed work
Configuration repository commitThe last successful sync plus pushThe same, plus anything that failed to push
Off-site archiveIts daily runUp to a day

Step 2: preserve what you are about to overwrite

Read-only / Safecapture the broken state first
# Substitute your own values before running:
ROUTER=edge-01.example.com
CASE=/srv/incident/CASE-1234

mkdir -p "$CASE"

ssh "vyos@$ROUTER" 'show configuration commands' > "$CASE/config-pre-restore.txt"
ssh "vyos@$ROUTER" 'show interfaces'          > "$CASE/interfaces-pre.txt"
ssh "vyos@$ROUTER" 'show ip route'            > "$CASE/routes-pre.txt"
ssh "vyos@$ROUTER" 'show ip bgp summary'      > "$CASE/bgp-pre.txt"
ssh "vyos@$ROUTER" 'show ip ospf neighbor'    > "$CASE/ospf-pre.txt"

The broken configuration is evidence. It is the only artefact that explains why the restore was needed, and load overwrites it in the candidate the moment you run it. The operational captures matter for a different reason: after the restore you will need to answer “is this router doing what it was doing before”, and that question needs a before.

Configuration changeand give yourself a way back
configure
save /config/backups/pre-restore.boot
exit

The archive records committed states only. If the box is sitting on a hand-repaired configuration that was never committed, or on a state you want back for comparison rather than for service, the pre-restore snapshot is the only thing that holds it. It costs one command and it is the difference between a reversible restore and a one-way one.

Step 3: get the file onto the router

Read-only / Safepush from the workstation - the router stays a server
# Substitute your own values before running:
ROUTER=edge-01.example.com
SRC=/srv/configs/edge-01-config.boot

scp "$SRC" "vyos@$ROUTER:/tmp/restore.boot"

ssh "vyos@$ROUTER" 'ls -la /tmp/restore.boot'

Step 4: load, then read the entire diff

Configuration changeload replaces the candidate; the active configuration is untouched
configure
load /tmp/restore.boot
compare
show configuration

load is not a merge. The candidate becomes the file, in full, and anything the file does not mention is gone from the candidate whether or not it was deliberate. The running configuration is untouched until commit, which is what makes this step reversible: discard at any point here leaves the router exactly as it was.

Now read compare properly. On a small change the diff is three lines and scanning it is enough. On a restore the diff is the whole drift since the snapshot, and it is the one moment when everything the restore will remove is visible in one place. The restore-lesson failure mode is precisely this: an operator loads a configuration from the repository, skips the verify step, commits, and discovers afterwards that the file carried a stale routing-protocol password. compare would have shown it.

Step 5: same appliance, or different?

This is where restores fail in ways that look like hardware faults.

Read-only / Safewhat the file expects versus what the box has
show interfaces
show configuration commands | match "interfaces ethernet"
show configuration commands | match "hw-id"

A VyOS configuration file names interfaces and, where it was written with them, pins each one to a MAC address with hw-id. Restored onto the same appliance that is invisible. Restored onto a replacement it is decisive: the new chassis has different MAC addresses, and possibly a different naming scheme entirely - eth0 on one platform, eno1 or ens4 on another.

Symptom after a cross-appliance restoreCause
An interface exists in the configuration but not in show interfacesThe file names an interface this hardware does not have
The interface exists but never comes uphw-id pins it to a MAC that is not on this chassis
Everything is up, nothing routesAddresses restored correctly, but the far side is cabled to a different port
Adjacencies never formThe addressing moved with the replacement and the peers still hold the old values

Fix these in the candidate before committing. Committing first and repairing afterwards means the router applies a configuration with no working interface - on a remote appliance, that is the console-only scenario, and on a fresh replacement there may not be a console configured yet.

Step 6: commit with a timer, verify from outside

Service impact possiblethe restore commit
compare
commit-confirm 15
exit

A full restore replaces the interface addressing, the SSH service binding, the management VRF and the firewall in a single commit. Every one of those can carry the session you are typing into. commit-confirm is not a formality here: it is the mechanism that returns the router to a reachable state without requiring you to still be connected to it.

Choose the timer for the verification you actually intend to run. Adjacencies take time to form - a BGP session with a full table is not a five-minute check - and a timer that fires while you are still verifying reverts a restore that was working.

Read-only / Safeverify in dependency order, outside configure mode
show interfaces
show ip route
show ip ospf neighbor
show ip bgp summary
show vpn ipsec sa

Interfaces before addresses, addresses before adjacencies, adjacencies before routes, routes before traffic. Verifying out of order produces confident wrong answers: a BGP session that is Idle because an interface never came up looks identical to a BGP session that is Idle because the password is stale, and only the ordering separates them.

Configuration changekeep it, then persist it
configure
confirm
save
exit

Step 7: prove it persisted

Read-only / Safethe two checks that catch the classic ending
ls -la /config/config.boot
diff /config/config.boot <(show configuration commands)
show configuration commit

save with no argument writes /config/config.boot. save <path> writes a snapshot somewhere else and leaves the boot file exactly as it was, which is the trap in Step 2: an operator who took a pre-restore snapshot has already typed save once that day and it is easy to believe the second one happened. An empty diff is the proof, and a restore is followed by a reboot more often than most changes are.

Then close the loop off-box. Push the restored configuration to the backup location and the repository. Otherwise the next scheduled sync captures the restored state as a large unexplained change, and the next person reading the repository history sees a rewrite with no commit message behind it.

Common patterns

SymptomLikely causeResolution
Restore reverted itself minutes latercommit-confirm fired before verification finishedRe-load and use a timer sized for adjacency formation
Restore gone after a rebootsave <path> ran, plain save did notRe-apply and save; confirm with an empty diff
Colleague change vanishedThe source predated itNot recoverable from the router. Re-apply from the repository history
Commit rejected at validator timeThe file came from a different release trainMigrate the configuration; do not retry the same load
Interface missing after a cross-appliance restorehw-id pins it to a MAC this chassis does not haveCorrect or remove hw-id in the candidate before committing
Everything up, no adjacenciesAddressing changed with the replacement; peers still hold the old valuesCoordinate with the peer, then re-check
load scp:// fails with a host-key errorThe router has never connected to the backup hostPush the file with scp from the workstation instead
Configuration restored, traffic still wrongVerified the configuration, not the FIBVerify at the protocol layer, in dependency order
Next backup shows a huge unexplained diffThe restored state was never pushed back off-boxReconcile the repository as the last step of the restore
Restore fixed the outage, compromise returnedThe file predated a credential rotationRe-apply every rotation after any restore during a security incident

Escalation

Escalate when:

  • The compare output shows other people work about to be discarded.
  • The router is one half of a redundant pair.
  • No available source is fresh enough to be useful.
  • The target hardware differs from the source appliance.
  • The restore is part of a suspected compromise.
  • The commit fails a validator, which means a release-train mismatch.
  • The console is the only remaining transfer path.

References

  1. VyOS documentation - configuration file and load
  2. VyOS documentation - basic system configuration
  3. VyOS documentation - rollback
  4. VyOS documentation - commit-confirm
  5. VyOS documentation - CLI fundamentals
  6. VyOS documentation - Ethernet interfaces