Runbook: Back Up a VyOS Configuration
1 · Prerequisites
Confirm every item is in place before any state change.
- Saved configuration — /config/config.boot, scp, off-box copy, versioning
- Remote backup — cron + rsync, version control commit, S3 bucket
- Candidate, active, saved — the three configurations every VyOS box has
- save / discard / exit — exiting configure mode with intent
- Configuration history — the archive of every successful commit
- save — making the configuration persist
- DR validation — quarterly DR drill, full restore, scenario test
- WireGuard keys — Curve25519 key pairs, preshared key, key management
2 · Pre-checks
Read-only diagnostic commands. If any of these don't match expected output, stop and investigate further.
- · The router is the one the backup will be filed under:
show configuration commands | match host-name - ·
/confighas free space and/config/archive/exists and is writable:df -h /configandls -la /config/archive/ - · The candidate is empty: from
configure, runcompareand expect no output. A diff here is somebody else unfinished change - the candidate survivesexitand the next operator inherits it - · The newest archive entry is accounted for:
show system commitnames a user, a source address and a timestamp for every entry. One you cannot explain means an unrecorded change is about to become the reference state - · You know which save you are running: bare
saveoverwrites/config/config.boot;save /config/backups/LABEL.bootwrites only that path and leaves the boot configuration untouched - · The destination for the off-box copy is access-controlled, because the file carries local user password hashes plus plaintext HTTPS API keys, IPsec pre-shared secrets and BGP session passwords
3 · Procedure
Execute each step in order. Verify the expected output of a step before moving to the next.
- 1Decide which backup this is - pre-change snapshot, pre-upgrade snapshot, or the routine off-box copy. The commands are the same; the destination and the retention are not.
- 2Capture read-only evidence first:
show configuration commands,show system commit,ls -la /config/archive/. This is the state you are claiming to have backed up. - 3Confirm the candidate is empty with
comparefromconfigure, and leave it empty. Do not commit a diff you did not author. - 4Preserve the existing boot file before anything writes to it:
sudo cp -a /config/config.boot /config/backups/config.boot.$(date +%Y%m%d-%H%M%S). This is the step that makes a mistakensavereversible when the archive cannot help. - 5Write the saved configuration:
savefrom configure mode for the boot file, orsave /config/backups/LABEL.bootfor a labelled snapshot.savewrites the RUNNING configuration, never the candidate. - 6Prove the saved file matches what is running:
configure,load /config/config.boot,compare. No output means they agree; any diff is exactly what the next reboot will discard. - 7Clear the candidate you just loaded:
discard. A loaded candidate left in place is the next operator surprise commit. - 8Copy the file off the router with
scp, then verify the copy withsha256sumon both ends. An unverified transfer is a file that looks like a backup until restore time. - 9Commit the off-box copy to version control and push. A local commit on the backup host is one location, not two.
- 10Treat the artefact as a secret, because on 1.5 it is one: the WireGuard private key and every PKI private key are values inside
config.boot. Confirm the destination repository and its access control are approved to hold key material before the first push, and useshow configuration commands | strip-privatefor anything that goes into a ticket. - 11Enumerate what is still outside the configuration on THIS router -
/config/auth/letsencrypt, anything under/config/scripts/, any file placed by hand - and back it up separately or record regenerate-on-restore in the DR plan. - 12Check chain freshness: the newest entry in
show system commitmust not be newer than the backup you just filed, and the remote repository must hold the push. - 13Attach the evidence - the checksum, the commit hash, the archive listing - to the change record, and confirm the restore drill for this router is inside the current DR cycle.
4 · Verification
Confirm the procedure actually fixed the problem.
- ✓From
configure,load /config/config.bootfollowed bycompareproduces no output: the saved file and the running configuration agree - ✓The candidate was cleared with
discardafter that check -compareis empty again and nothing is waiting to be committed - ✓
sha256sumof the off-box copy matchessha256sum /config/config.boottaken on the router - ✓The commit exists on the REMOTE, not only on the backup host:
git log origin/main -1names it - ✓The newest entry in
show system commitis no newer than the backup just filed - ✓The backup destination is approved to hold private keys, because
config.bootcontains the WireGuard private key and the PKI private keys on this release - ✓Anything still outside the configuration on this router is either backed up separately or explicitly recorded as regenerate-on-restore in the DR plan
- ✓The change record carries the checksum, the commit hash and the archive listing, so a later restore can prove which state it is restoring
5 · Rollback
If verification fails, undo the procedure in reverse order.
- ↶A mistaken bare
saveis reversible while the intended state is still in the commit archive:configure,rollback Nfor the entry named inshow system commit,compare,commit,save - ↶If the previous
/config/config.bootwas never a running configuration - it was copied in rather than committed - the archive does not contain it. The only copy is the one taken in step 4; restore from that:sudo cp -a /config/backups/config.boot.STAMP /config/config.boot - ↶Verify any restored boot file the same way it was verified when written:
configure,load /config/config.boot,compare, thendiscard. Do not rely on a file you have not read - ↶A loaded candidate left behind by the verification step is cleared with
discard. This costs nothing and is not optional - ↶POINT OF NO RETURN: a configuration pushed to a repository, bucket or ticket that should not hold it cannot be recalled. Deleting the file in a later commit does not remove it from history, mirrors or CI caches. Rotate every secret in the file and raise a credential exposure
- ↶POINT OF NO RETURN: an archive entry dropped by
system config-management commit-revisionspruning is gone. Raise retention before a change-heavy window, not after
6 · Escalation
When the runbook isn't enough, contact:
- · Escalate to security the moment a configuration file reaches a destination whose access control you cannot describe - the file is a credential artefact, not a text file
- · Escalate to the platform team if
df -h /configshows no free space or/config/archive/is not writable:saveand the commit archive both write there - · Escalate to the peer or provider owner before planning regeneration of WireGuard or IPsec key material - the new public key has to reach every peer, and that is their change window, not yours
- · Escalate to network engineering if this router is one of a redundant pair and the backup is the first step of a failover: the consistent thing has to be the pair, not this file
- · Escalate to the change owner if the newest archive entry names a user or source address that is not on the change record
- · Escalate to whoever owns the backup chain if the newest off-box copy or the newest repository commit is older than the newest archive entry on the router - the chain is broken and has been for a while
Backing up a VyOS router is two operations that get talked about as one.
save writes the running configuration to /config/config.boot - on the
router, on the disk that is the thing most likely to fail. The backup is the
copy that survives the router: off the box, versioned, and held somewhere whose
access control matches the fact that the file contains every secret the router
knows.
This runbook is the second operation, with the first done carefully enough that it does not poison the second. Most of the work is not the copy. It is proving three things the file itself will not tell you: whether it matches what is actually running, whether it is complete, and whether it has ever been restored.
When to use this runbook
- Before a change whose rollback plan needs a named starting point.
- Before an image upgrade.
- When the scheduled off-box chain has stopped and you are catching up by hand.
- When a router has been built or rebuilt and has no entry in the chain yet.
When not to use it
- You are restoring a configuration - that is
vyos-rb-restore-vyos-config. - The router is gone - that is
vyos-rb-recover-failed-router. - You want to undo the last change - that is
vyos-rb-rollback-bad-config. The commit archive already holds the previous state; taking a fresh backup first adds nothing and delays the fix.
Blast radius
Read-only except for one write. Bare save overwrites /config/config.boot,
which is what the router loads on its next boot. Nothing about the running
configuration changes, so nothing about forwarding changes today - the cost of
getting this wrong is paid at the next reboot, which may be months away and
will not be attended by the person who made the mistake.
What a VyOS backup has to contain
There are three things on the router that a restore needs, and only the first is what people mean when they say “the config”.
/config/config.boot is the saved configuration: a plain-text rendering of
the configuration tree in config-file syntax. It is written by save and read
on boot. If it is missing, the router boots empty.
/config/archive/ is the commit archive - one gzipped snapshot per
successful commit, rotated by logrotate as config.boot.1.gz,
config.boot.2.gz and so on, alongside a commits log and the
config.boot-rollback / config.boot-prerollback working files. It is history
that the boot file does not have, and it is what rollback N and compare N
read. Retention defaults to 20 entries and is set with
system config-management commit-revisions. Read it with show system commit
rather than by listing the directory - the numbers in that listing are the
revision numbers the rollback command takes.
/config/config.boot contains secrets on this release. On VyOS 1.4 and
later, key material that used to sit in files moved into the configuration:
- The WireGuard private key is a base64 value under
interfaces wireguard wgN private-key. It is not a file path, and there has never been aprivate-key fileform; VyOS 1.3 named a key pair whose material lived under/config/auth/wireguard/NAME/, and theinterfaces/25-to-26migrator read that file and wrote its contents into the CLI. - CA and certificate private keys, DH parameters, generic key pairs, OpenSSH
private keys and OpenVPN shared secrets all live under the
pkinode, inline.
So the backup problem is the reverse of what older runbooks describe. The configuration is not missing the keys - it is the keys, and every copy of it is a copy of them.
/config/auth/ still exists and still holds material the configuration
references by path rather than containing: /config/auth/letsencrypt is
certbot’s directory for ACME account and certificate data, and it is where
generate pki ... file writes when asked for a file rather than an install.
Anything an operator placed there by hand is in the same position. Treat it as a
second artefact to enumerate on this specific router rather than assuming a
fixed list.
Step 0: which save are you running
The two forms of save do different things, and confusing them is the single
most common way a backup silently fails.
| Command | Writes | Leaves /config/config.boot |
|---|---|---|
save | /config/config.boot | overwritten |
save /config/backups/LABEL.boot | that path only | untouched |
The trap runs in both directions. An operator who takes a labelled snapshot and
believes the live configuration is now persistent has persisted nothing: the
next reboot still loads the old boot file. An operator who types bare save
during what they thought was a snapshot has replaced the boot configuration
with whatever is running right now.
Step 1: evidence before anything is written
show configuration commands
show system commit
ls -la /config/archive/
df -h /configshow configuration commands prints the tree in set form - the shape a
reviewer can diff and an auditor can replay line by line. show configuration commit prints the archive listing with a commit number, a timestamp, a user
and the source address for each entry.
Read that listing before you go further. If the newest entry names a user or a source address that is not on the change record, somebody made an unrecorded change and you are about to promote it to the reference state. That is an escalation, not a note.
Step 2: the candidate is not yours
configure
compareThe candidate configuration lives in configd’s memory and survives exit. The
next operator to type configure resumes it exactly where it was left. So an
empty compare is not a formality - it is the check that the change you are
about to make permanent is the one you can describe.
If compare shows a diff you did not author, stop. Do not discard it either;
somebody is building that change. Find them.
Step 3: make the save reversible before you save
The commit archive is the usual safety net, and it is a good one: every
configuration that has ever been running has an entry. But it only covers
configurations that arrived by commit. A /config/config.boot that was written
some other way - copied in by a restore, pulled from the repository, edited by
hand - has no archive entry, and bare save will overwrite it with nothing to
fall back to.
STAMP=$(date +%Y%m%d-%H%M%S)
sudo mkdir -p /config/backups
sudo cp -a /config/config.boot "/config/backups/config.boot.$STAMP"
ls -la /config/backups/This costs a few kilobytes and two seconds. Skip it only when you have just
read show system commit and can name the archive entry that holds the
current boot state.
Step 4: save
save
save /config/backups/before-upgrade.bootsave writes the running configuration. Not the candidate. If you are
halfway through building a change, save persists the state as it was before
your edits, which is usually exactly what a pre-change snapshot should be - and
is silently wrong if you assumed it captured your work.
Step 5: prove the saved file matches what is running
This is the step almost nobody performs, and it is the one that turns a file into a backup.
configure
load /config/config.boot
compare
discard
exitload replaces the candidate with the contents of the file and leaves the
running configuration alone. compare then diffs candidate against running. No
output means the saved file and the running configuration agree. Any output is
precisely the set of directives the next reboot will discard - usually a change
that was committed and never saved.
Step 6: off the box, and prove the copy arrived intact
# Substitute your own values before running:
ROUTER=192.0.2.1
NAME=edge-01
scp "vyos@$ROUTER:/config/config.boot" "./configs/$NAME-config.boot"
ssh "vyos@$ROUTER" sha256sum /config/config.boot
sha256sum "./configs/$NAME-config.boot"The checksum is not ceremony. A truncated transfer produces a file that has the right name, roughly the right size, and a partial configuration tree. It looks like a backup in every listing you will ever run against it, and it fails the config-file parser at the one moment you need it - during the outage, on the replacement hardware, with the change window running.
Step 7: version it, and understand what you just published
# Substitute your own values before running:
NAME=edge-01
cd ./configs
git add "$NAME-config.boot"
git commit -m "$NAME: pre-change snapshot"
git push origin main
git log origin/main -1A single off-box copy is one file at one point in time; the next copy overwrites it. The repository is what makes a configuration from six months ago recoverable. The last line matters as much as the push: a cron job that commits locally and never pushes leaves the entire history on one host, which is the host most likely to be rebuilt.
Step 8: freshness, and whether the chain has ever been restored
Two questions close this runbook out, and a backup that fails either one is worse than no backup, because it produces confidence.
Is the chain current? Compare the newest entry in show configuration commit on the router against the newest commit in the repository. If the
router has commits the repository has never seen, the chain broke at some point
in the past and nobody noticed - the usual causes are a task-scheduler entry
that stopped running, an SSH host key that changed after a backup-host rebuild,
or a cron job that commits without pushing.
Has it ever been restored? A disaster recovery procedure that has never
been executed is a hypothesis. The drill is load the backup onto a lab router,
compare it, commit it, and verify that the routing adjacencies come up -
and then record how long that actually took, because the measured recovery time
is the only honest input to an SLA conversation.
Common patterns
| Symptom | Likely cause | Resolution |
|---|---|---|
| Change gone after an unrelated reboot | commit without save | Re-apply, then save. The archive holds the lost change; rollback N finds it |
save reported success, reboot still reverted | save /config/backups/x.boot was used; the boot file was never touched | Run bare save as well |
| Off-box file is current, repository is a month behind | The job commits locally and never pushes | Add the push; alert on push failure |
| Backup job stopped after a backup-host rebuild | SSH host key changed - Host key verification failed | Update known_hosts; add the step to the host rebuild procedure |
| Restore onto replacement hardware fails validation | Interface names differ on the new platform | Map old names to new before commit; see vyos-rb-restore-vyos-config |
| Tunnels down after an otherwise clean restore | On 1.5 the WireGuard and PKI keys are IN the configuration, so this is not missing key material - check for a peer-side key rotation that this saved configuration predates | Compare show interfaces wireguard wgN public-key against the peer entry on the far router, and correct whichever end is stale |
load /config/config.boot then compare shows a diff | A committed change was never saved | Decide which state is correct, then save - or rollback and save |
| The archive entry you wanted is not in the listing | Pruned by commit-revisions retention | Raise retention before change-heavy windows; recover from the repository |
Escalation
Escalate when:
- The configuration reached a destination whose access control you cannot describe.
/configis full, or/config/archive/is not writable.- The chain is behind the router and you cannot establish when it broke.
- Key material has to be regenerated, which is a change for every peer.
- The newest archive entry names somebody who is not on the change record.
References
- VyOS documentation - Configuration file
- VyOS documentation - Basic system configuration
- VyOS documentation - Task scheduler
- rsync documentation
- git-commit documentation
- NIST SP 800-34 Rev. 1 - Contingency Planning Guide
- vyos-1x, circinus branch - python/vyos/config_mgmt.py (archive paths and revision file naming)
- vyos-1x, circinus branch - interface-definitions/pki.xml.in (the PKI material held inline in the configuration)