Runbook: Restore a VyOS Configuration
1 · Prerequisites
Confirm every item is in place before any state change.
- Saved configuration — /config/config.boot, scp, off-box copy, versioning
- Remote backup — cron + rsync, version control commit, S3 bucket
- Restore onto the same appliance — load saved, verify, commit, save
- Restore onto a different appliance — hardware replacement, configuration migration, IP changes
- Full router loss — RMA, replacement, restore, verify adjacency
- commit-confirm — the rollback safety net for remote changes
- Configuration history — the archive of every successful commit
- save — making the configuration persist
- Out-of-band access — OOB management VRF, console server, IPMI, the lockout safety net
2 · Pre-checks
Read-only diagnostic commands. If any of these don't match expected output, stop and investigate further.
- · The question "is this a restore or a rollback?" has been answered. If the state you want is an entry in
show configuration commit, this is the wrong runbook -rollback Nneeds no file, no transfer and no source-selection decision - · The source file is named exactly, with its age: an archive entry,
/config/backups/<label>.boot, the off-box rsync copy, or a specific commit in the configuration repository. "The backup" is not a source; each of those has a different lag and restoring from the wrong one discards different work - · The gap between the source and now is understood and accepted by the change owner. Everything committed after the snapshot is about to be discarded, including changes made by people who are not in this incident
- · Out-of-band access is confirmed and tested. A restored configuration replaces the interface addressing, the SSH binding and the firewall in one commit, so it can remove the session running it
- · Whether the target hardware matches the hardware the file was written for is established: same interface names, same
hw-idvalues, same addressing. A configuration is not portable between appliances by default - · The current configuration has been captured off-box first, even though it is the broken one.
loadoverwrites the candidate, and the broken configuration is the only record of what went wrong - · A pre-restore snapshot exists on the box:
save /config/backups/pre-restore.boot. The archive only records committed states, so an uncommitted or partially repaired candidate has no other way back - · For a restore that touches the routing plane, the peers or the change owner know it is happening. A restore re-establishes adjacencies from scratch and the reconvergence is visible from other people routers
3 · Procedure
Execute each step in order. Verify the expected output of a step before moving to the next.
- 1Decide whether this is a restore at all.
show configuration commitlists every archived commit with its timestamp, user and source address; if the good state is in that list, stop here and userollback Ninstead. - 2Choose the source deliberately and write down its age. The local archive is as fresh as the last commit,
/config/backups/as fresh as the last manual snapshot, the off-box rsync copy as fresh as its interval, and the off-site archive as fresh as its daily run. - 3Capture the state you are about to destroy:
show configuration commandswritten off-box, plus the operational view -show interfaces,show ip route,show ip bgp summary,show ip ospf neighbor- so the post-restore comparison has a baseline. - 4Take a pre-restore snapshot on the box with
save /config/backups/pre-restore.boot, so a restore that turns out to be wrong has somewhere to go back to that is not the archive. - 5Get the file onto the router by the path most likely to work.
load scp://...makes the ROUTER the client and needs its DNS, routing and host keys intact; during an outage those are candidates for the thing that is broken. Pushing the file withscpfrom the workstation, or pasting it over the console, does not. - 6Load it into the candidate:
configure, thenload /path/to/file. This REPLACES the candidate; it does not merge. The active configuration is untouched untilcommit. - 7Read the whole
compareoutput, not the part you expected. On a restore the diff is every change made since the snapshot was taken, which includes other people work, and it is the only place that becomes visible. - 8If the hardware differs from the source, fix the hardware-bound nodes in the candidate BEFORE committing: interface names,
hw-idvalues, and any address that has moved. Committing first and fixing afterwards means the box comes up with no working interface. - 9Commit with
commit-confirmwhenever the restore includes the interface, SSH, VRF or firewall configuration that carries your session - which for a full restore is always. Then verify from OUTSIDE configure mode, andconfirmonly after the check passes. - 10
savewith no argument.save /config/backups/...writes a snapshot and leaves/config/config.bootuntouched, so a restore that ends that way is one reboot away from reverting. - 11Verify at the protocol layer, in dependency order: interfaces up, then addresses, then adjacencies, then routes, then traffic. A configuration that matches the file proves the load worked, not that the router does.
- 12Reconcile the sources of truth: push the restored configuration back to the off-box copy and the repository, or the next scheduled backup captures the restored state as if it were a change nobody made.
4 · Verification
Confirm the procedure actually fixed the problem.
- ✓
comparewas read in full before the commit, and every line in it was either expected or explicitly accepted as lost work - ✓
show interfaceslists every interface the configuration expects, with the expected addresses and admin state - not merely the same count - ✓No interface is missing because its
hw-iddoes not exist on this hardware:show configuration commands | match "hw-id"matches what the box actually has - ✓Routing adjacencies are re-established rather than merely configured:
show ip ospf neighborshows Full,show ip bgp summaryshows Established with a plausible prefix count, and any BFD sessions are up - ✓
show ip routecontains the expected routes, including the default, and the FIB agrees with the protocol tables - ✓Traffic passes end to end through the router, tested from a client rather than from the router itself
- ✓The restore persisted:
ls -la /config/config.bootshows a current timestamp anddiff /config/config.boot <(show configuration commands)returns nothing - ✓A new archive entry exists for the restore commit in
show configuration commit, attributed to the operator who ran it - ✓The off-box backup and the configuration repository have been updated with the restored state, so the next scheduled run does not report it as unexplained drift
5 · Rollback
If verification fails, undo the procedure in reverse order.
- ↶Before the commit,
discardthrows the loaded candidate away and leaves the running configuration untouched. This is the free exit and it is available right up to the moment you commit - ↶While the
commit-confirmtimer runs, doing nothing restores the pre-restore state automatically. This is the intended path when the verification fails, and it is why a restore of the access path is committed no other way - ↶After a confirmed restore, the way back is the pre-restore snapshot:
load /config/backups/pre-restore.boot,compare,commit,save. This is why Step 4 exists - ↶
rollback 1also works after a confirmed restore, because the restore itself wrote an archive entry - but readcomparefirst. During an incident the previous entry may be the broken configuration you were escaping - ↶POINT OF NO RETURN: work committed after the snapshot was taken. A restore discards it silently; nothing in the output names what was lost, which is why the
compareoutput is the record - ↶POINT OF NO RETURN: an archive entry that has been pruned.
commit-revisionsdefaults to 20 and pruning is irreversible, so an older state exists only in the off-box copy or nowhere - ↶POINT OF NO RETURN: a restore performed as part of a credential incident. Every pre-incident file contains the compromised secrets, so the restore must be followed by re-applying every rotation
- ↶A reboot is not a rollback. If
savehas already run, the box comes back on the restored configuration - the pre-restore state is only in the snapshot file and the archive
6 · Escalation
When the runbook isn't enough, contact:
- · Escalate to the change owner before discarding work: the
compareoutput names people whose changes are about to be reverted, and that is their decision as much as yours - · Escalate to network engineering before restoring a router in a redundant pair. A restore drops and re-establishes every adjacency, so the pair is briefly running on one member
- · Escalate to the platform team if no source is fresh enough to be useful. A backup that predates a major change is a decision about how much to rebuild by hand, not a restore
- · Escalate if the target hardware differs from the source: different interface names, a different port count, or different addressing turn a restore into a migration with its own review
- · Escalate to security if the reason for the restore is suspected compromise. Restoring from a pre-incident file reintroduces whatever was disclosed, and that sequencing is not an on-call decision
- · Escalate if the load fails on a validator. A configuration written by a different VyOS release train can be rejected by the current schema, and the fix is a migration rather than a retry
- · Escalate if the router cannot reach the backup source and the console is the only remaining path - a large configuration pasted over a serial line is slow enough to need its own window
A restore feels like an undo and is not one. rollback N is an undo: it moves
the router back along its own history, one archived commit at a time, and the
box has never been anything other than itself. A restore takes a file written
at some other moment, possibly for some other appliance, and replaces the
entire candidate configuration with it. Everything between the file and now
disappears in one commit, and nothing in the output tells you what.
That is the whole difficulty. load is a single command that always succeeds
if the file parses, and the three decisions that determine whether the restore
helps or hurts all happen before it: whether this should be a rollback instead,
which source you picked and therefore how much recent work you are discarding,
and whether the hardware under the file is the hardware the file was written
for.
The commands here are short. The reading is long, and the compare output is
where the work is.
When to use this runbook
- The saved configuration is lost or corrupted and the router is otherwise running.
- A replacement appliance needs the failed one configuration on it.
- A change went wrong and the state you need is older than the archive holds - because it was pruned, or because the box was reinstalled.
- A disaster-recovery drill is exercising the restore path deliberately.
When not to use it
- The good state is an entry in
show configuration commit. Usevyos-rb-rollback-bad-config: no file, no transfer, no staleness decision, and the diff is small enough to read properly. - You are taking the backup rather than using one - that is
vyos-rb-backup-vyos-config. - The router is dead and this is a full rebuild including installation and
hardware. The restore is one stage of
vyos-rb-recover-failed-router, and the stages before it decide whether you are restoring at all. - The reason for the restore is a credential compromise. Restoring reintroduces
every secret in the file: run
vyos-rb-respond-compromised-credentialsfirst, and treat the restore as a step inside it.
Blast radius
A full restore is the largest single change a VyOS router accepts. It replaces interfaces, addressing, routing protocols, firewall, NAT and the management plane in one commit. Every adjacency drops and re-forms. Every stateful firewall connection is re-evaluated against a rule set the kernel has just been handed. If the router is in a redundant pair, the pair is running on one member for the duration.
It also, and less obviously, discards work. The blast radius includes every change committed after the snapshot was taken, by anyone, for any reason.
Step 0: is this a restore or a rollback?
show configuration commit
show system config-managementshow configuration commit lists each archived commit with its number,
timestamp, user and source address. If the state you want appears in that list,
this runbook is the expensive way to get it.
| Situation | Right tool |
|---|---|
| Good state is a recent archive entry | rollback N - vyos-rb-rollback-bad-config |
| Good state predates the retained archive | Restore from a file. This runbook |
Archive is intact but /config/config.boot is corrupt | Restore from the archive entry, then save |
| Box was reinstalled - archive is empty | Restore from the off-box copy or the repository |
| Different appliance entirely | Restore, plus hardware adaptation. Step 5 |
The archive default retention is 20 entries. On a busy router that can be a single afternoon, which is exactly why a restore is sometimes the only option for a state that feels recent.
Step 1: choose the source, and name its lag
Every backup location trades freshness for independence, and the restore inherits whichever trade you pick.
| Source | Fresh as of | What it costs you |
|---|---|---|
/config/archive/ entry | The last commit | Nothing, but it dies with the disk |
/config/backups/<label>.boot | Whenever someone last ran save <path> | Manual, so often much older than it looks |
| Off-box copy from the scheduled sync | Its interval | Up to one interval of committed work |
| Configuration repository commit | The last successful sync plus push | The same, plus anything that failed to push |
| Off-site archive | Its daily run | Up to a day |
Step 2: preserve what you are about to overwrite
# Substitute your own values before running:
ROUTER=edge-01.example.com
CASE=/srv/incident/CASE-1234
mkdir -p "$CASE"
ssh "vyos@$ROUTER" 'show configuration commands' > "$CASE/config-pre-restore.txt"
ssh "vyos@$ROUTER" 'show interfaces' > "$CASE/interfaces-pre.txt"
ssh "vyos@$ROUTER" 'show ip route' > "$CASE/routes-pre.txt"
ssh "vyos@$ROUTER" 'show ip bgp summary' > "$CASE/bgp-pre.txt"
ssh "vyos@$ROUTER" 'show ip ospf neighbor' > "$CASE/ospf-pre.txt"The broken configuration is evidence. It is the only artefact that explains why
the restore was needed, and load overwrites it in the candidate the moment
you run it. The operational captures matter for a different reason: after the
restore you will need to answer “is this router doing what it was doing
before”, and that question needs a before.
configure
save /config/backups/pre-restore.boot
exitThe archive records committed states only. If the box is sitting on a hand-repaired configuration that was never committed, or on a state you want back for comparison rather than for service, the pre-restore snapshot is the only thing that holds it. It costs one command and it is the difference between a reversible restore and a one-way one.
Step 3: get the file onto the router
# Substitute your own values before running:
ROUTER=edge-01.example.com
SRC=/srv/configs/edge-01-config.boot
scp "$SRC" "vyos@$ROUTER:/tmp/restore.boot"
ssh "vyos@$ROUTER" 'ls -la /tmp/restore.boot'Step 4: load, then read the entire diff
configure
load /tmp/restore.boot
compare
show configurationload is not a merge. The candidate becomes the file, in full, and anything
the file does not mention is gone from the candidate whether or not it was
deliberate. The running configuration is untouched until commit, which is
what makes this step reversible: discard at any point here leaves the router
exactly as it was.
Now read compare properly. On a small change the diff is three lines and
scanning it is enough. On a restore the diff is the whole drift since the
snapshot, and it is the one moment when everything the restore will remove is
visible in one place. The restore-lesson failure mode is precisely this: an
operator loads a configuration from the repository, skips the verify step,
commits, and discovers afterwards that the file carried a stale routing-protocol
password. compare would have shown it.
Step 5: same appliance, or different?
This is where restores fail in ways that look like hardware faults.
show interfaces
show configuration commands | match "interfaces ethernet"
show configuration commands | match "hw-id"A VyOS configuration file names interfaces and, where it was written with them,
pins each one to a MAC address with hw-id. Restored onto the same appliance
that is invisible. Restored onto a replacement it is decisive: the new chassis
has different MAC addresses, and possibly a different naming scheme entirely -
eth0 on one platform, eno1 or ens4 on another.
| Symptom after a cross-appliance restore | Cause |
|---|---|
An interface exists in the configuration but not in show interfaces | The file names an interface this hardware does not have |
| The interface exists but never comes up | hw-id pins it to a MAC that is not on this chassis |
| Everything is up, nothing routes | Addresses restored correctly, but the far side is cabled to a different port |
| Adjacencies never form | The addressing moved with the replacement and the peers still hold the old values |
Fix these in the candidate before committing. Committing first and repairing afterwards means the router applies a configuration with no working interface - on a remote appliance, that is the console-only scenario, and on a fresh replacement there may not be a console configured yet.
Step 6: commit with a timer, verify from outside
compare
commit-confirm 15
exitA full restore replaces the interface addressing, the SSH service binding, the
management VRF and the firewall in a single commit. Every one of those can
carry the session you are typing into. commit-confirm is not a formality
here: it is the mechanism that returns the router to a reachable state without
requiring you to still be connected to it.
Choose the timer for the verification you actually intend to run. Adjacencies take time to form - a BGP session with a full table is not a five-minute check - and a timer that fires while you are still verifying reverts a restore that was working.
show interfaces
show ip route
show ip ospf neighbor
show ip bgp summary
show vpn ipsec saInterfaces before addresses, addresses before adjacencies, adjacencies before routes, routes before traffic. Verifying out of order produces confident wrong answers: a BGP session that is Idle because an interface never came up looks identical to a BGP session that is Idle because the password is stale, and only the ordering separates them.
configure
confirm
save
exitStep 7: prove it persisted
ls -la /config/config.boot
diff /config/config.boot <(show configuration commands)
show configuration commitsave with no argument writes /config/config.boot. save <path> writes a
snapshot somewhere else and leaves the boot file exactly as it was, which is
the trap in Step 2: an operator who took a pre-restore snapshot has already
typed save once that day and it is easy to believe the second one happened.
An empty diff is the proof, and a restore is followed by a reboot more often
than most changes are.
Then close the loop off-box. Push the restored configuration to the backup location and the repository. Otherwise the next scheduled sync captures the restored state as a large unexplained change, and the next person reading the repository history sees a rewrite with no commit message behind it.
Common patterns
| Symptom | Likely cause | Resolution |
|---|---|---|
| Restore reverted itself minutes later | commit-confirm fired before verification finished | Re-load and use a timer sized for adjacency formation |
| Restore gone after a reboot | save <path> ran, plain save did not | Re-apply and save; confirm with an empty diff |
| Colleague change vanished | The source predated it | Not recoverable from the router. Re-apply from the repository history |
| Commit rejected at validator time | The file came from a different release train | Migrate the configuration; do not retry the same load |
| Interface missing after a cross-appliance restore | hw-id pins it to a MAC this chassis does not have | Correct or remove hw-id in the candidate before committing |
| Everything up, no adjacencies | Addressing changed with the replacement; peers still hold the old values | Coordinate with the peer, then re-check |
load scp:// fails with a host-key error | The router has never connected to the backup host | Push the file with scp from the workstation instead |
| Configuration restored, traffic still wrong | Verified the configuration, not the FIB | Verify at the protocol layer, in dependency order |
| Next backup shows a huge unexplained diff | The restored state was never pushed back off-box | Reconcile the repository as the last step of the restore |
| Restore fixed the outage, compromise returned | The file predated a credential rotation | Re-apply every rotation after any restore during a security incident |
Escalation
Escalate when:
- The
compareoutput shows other people work about to be discarded. - The router is one half of a redundant pair.
- No available source is fresh enough to be useful.
- The target hardware differs from the source appliance.
- The restore is part of a suspected compromise.
- The commit fails a validator, which means a release-train mismatch.
- The console is the only remaining transfer path.