Objective
Taking a backup on VyOS is one command and it is not the subject of this lab. The subject is the gap between the four things people call “the backup” and what each of them can actually recover, because that gap is invisible until the day you need it and then it is the whole incident.
By the end you will have produced all four — /config/config.boot, the commit
archive, a labelled on-box snapshot, and an off-box copy — and you will have
watched each of them fail at something. You will also have run the restore that
is genuinely hard: onto a different chassis, where the configuration is correct,
the commit is clean, every interface is up, the adjacency count is right, and
no traffic passes.
That last failure is the one worth the lab time. It has no error message, no counter, and no log line. The only thing that separates it from a successful recovery is a table of MAC addresses somebody wrote down before the outage.
Architecture
Four VyOS VMs on four isolated bridges. edge-01 is the router in service;
peer is the OSPF neighbour and holds the far service; ops is the LAN host
that runs the end-to-end test and doubles as the backup target; edge-02 is
the replacement chassis, and it exists from the first task even though nothing
uses it until Task 6.
peer
+--------------------+
| eth0 MGMT .13 |
| eth1 203.0.113.2 |
| dum0 203.0.113.200|
+--------------------+
|
TRANSIT 203.0.113.0/30
|
+--------------------+
| edge-01 |
| eth0 MGMT .11 |
| eth1 203.0.113.1 |
| eth2 198.51.100.1 |
+--------------------+
|
LAN 198.51.100.0/24
|
+--------------------+
| ops |
| eth0 MGMT .20 |
| eth1 198.51.100.20|
+--------------------+
+--------------------+
| edge-02 | spare chassis
| eth0 MGMT .12 | NICs attached in a
| eth1 (LAN bridge) | DIFFERENT order --
| eth2 (TRANSIT) | this is deliberate
+--------------------+
| Segment | Prefix | edge-01 | edge-02 | peer | ops |
|---|---|---|---|---|---|
| MGMT | 192.0.2.0/24 | eth0 · .11 | eth0 · .12 | eth0 · .13 | eth0 · .20 |
| TRANSIT | 203.0.113.0/30 | eth1 · .1 | eth2 | eth1 · .2 | — |
| LAN | 198.51.100.0/24 | eth2 · .1 | eth1 | — | eth1 · .20 |
| Far service | 203.0.113.200/32 | — | — | dum0 | — |
Every address comes from a range RFC 5737 reserves for documentation, so nothing here can collide with a real network if a bridge is attached to something it should not be.
Read the edge-02 row again, because it is the design decision the whole
second half of the lab rests on. On edge-02 the vNIC that lands on the LAN
bridge is eth1, and on edge-01 the LAN is eth2. That is not a mistake in
the table. It is the thing a replacement chassis does to you when the ports are
cabled by their labels rather than by their bus positions, and it is what
Part LV-04 means by “the interface names may differ”.
Requirements
- A hypervisor with roughly 4 GiB of free RAM and 40 GiB of free disk —
four VMs at 1 GiB and 8 GiB each is the floor, and
edge-02gets rebuilt from scratch in Task 8. - The VyOS 1.5 LTS ISO.
compare saved,commit-confirmand thesystem config-managementtree are all written against 1.5. - Four isolated layer-2 bridges with no physical port: MGMT, TRANSIT, LAN, and one spare you will re-attach in Task 7.
- Console access to all four VMs, per the callout above.
- The ability to attach and detach vNICs and choose their order on the hypervisor. Task 7 needs it.
- No Internet access. Nothing here resolves DNS or reaches a public address.
- Roughly two and a half hours, of which perhaps twenty minutes is waiting on reboots and adjacency timers.
Scenario
edge-01 has been in service for two years. It carries one OSPF adjacency, one
LAN, and a forward filter that a security review added eight months ago. The
person who built it has left. What exists is a nightly copy of its
configuration on the ops host, a commit archive on the router itself, and
a general belief that “we have backups”.
This week the chassis has started logging correctable memory errors and a replacement has arrived. Your job is to prove the belief before you need it: establish which backup can rebuild this router, on which hardware, losing how much recent work — and then actually do the rebuild while nothing is on fire.
Tasks
Configuration blocks are written for the [edit] prompt. Where a block opens
with configure and you are already at [edit], skip that line. Blocks
opening with set or delete continue the session you have open. Blocks
tagged bash are the operational-mode shell or the hypervisor host, as their
comments say. Blocks made up of show commands are operational
mode; from inside configure, prefix each one with run.
Task 1 — Build the estate, and write down the chassis identity table
Install four routers from the ISO. Give each one a hostname and its MGMT
address on eth0, and stop there for a moment — before any lab configuration,
capture what each chassis physically is.
# Run from operational mode on edge-01, then on edge-02.
JOURNAL="$HOME/lab22"
mkdir -p "$JOURNAL"
for i in eth0 eth1 eth2; do
printf '%-6s %s\n' "$i" "$(cat /sys/class/net/$i/address)"
done | tee "$JOURNAL/chassis-identity.txt"
That file is the deliverable this lab is built around. Three lines per chassis, name and MAC, and the bridge each vNIC is attached to — which the hypervisor knows and the router does not, so write it in by hand.
Everything after Task 5 is either easy or impossible depending on whether that table exists. In production it is the single cheapest artefact in a disaster-recovery plan and the one most reliably missing.
Now build edge-01 as the router in service:
configure
set system host-name edge-01
set system name-server 192.0.2.13
set system time-zone UTC
set system config-management commit-revisions 20
set interfaces ethernet eth0 address 192.0.2.11/24
set interfaces ethernet eth0 description 'MGMT'
set interfaces ethernet eth1 address 203.0.113.1/30
set interfaces ethernet eth1 description 'TRANSIT to peer'
set interfaces ethernet eth2 address 198.51.100.1/24
set interfaces ethernet eth2 description 'LAN'
set protocols ospf parameters router-id 192.0.2.11
set protocols ospf area 0 network 203.0.113.0/30
set protocols ospf area 0 network 198.51.100.0/24
set protocols ospf interface eth2 passive
set firewall global-options state-policy established action accept
set firewall global-options state-policy related action accept
set firewall ipv4 forward filter default-action drop
set firewall ipv4 forward filter rule 10 action accept
set firewall ipv4 forward filter rule 10 description 'LAN to the far service'
set firewall ipv4 forward filter rule 10 inbound-interface name eth2
set firewall ipv4 forward filter rule 10 outbound-interface name eth1
set firewall ipv4 forward filter rule 10 state new
commit
save
peer holds the far service and the other end of the adjacency:
configure
set system host-name peer
set interfaces ethernet eth0 address 192.0.2.13/24
set interfaces ethernet eth1 address 203.0.113.2/30
set interfaces dummy dum0 address 203.0.113.200/32
set interfaces dummy dum0 description 'far service'
set protocols ospf parameters router-id 192.0.2.13
set protocols ospf area 0 network 203.0.113.0/30
set protocols ospf area 0 network 203.0.113.200/32
commit
save
ops is the LAN host, the end-to-end test source, and the backup target:
configure
set system host-name ops
set interfaces ethernet eth0 address 192.0.2.20/24
set interfaces ethernet eth1 address 198.51.100.20/24
set protocols static route 0.0.0.0/0 next-hop 198.51.100.1
commit
save
edge-02 gets a hostname and a MGMT address and nothing else. It is a bare
chassis, which is exactly what a replacement is.
Now establish the baseline from ops, not from edge-01. This distinction
runs through the whole lab: a router can reach its own neighbour while
forwarding nothing.
$ ping 203.0.113.200 count 3PING 203.0.113.200 (203.0.113.200) 56(84) bytes of data.
64 bytes from 203.0.113.200: icmp_seq=1 ttl=63 time=1.02 ms
64 bytes from 203.0.113.200: icmp_seq=2 ttl=63 time=0.88 ms
64 bytes from 203.0.113.200: icmp_seq=3 ttl=63 time=0.91 msIllustrative output
On edge-01, record the four facts every later task compares against: the
adjacency, the route, the interface set, and the configuration itself.
show ip ospf neighbor
show ip route
show interfaces
show configuration commands
Task 2 — What save persists, and the two ways a backup can lie
Both of these are one-line mistakes with a delayed consequence, which is why they survive code review and turn up during a recovery instead.
The first lie: committed, never saved. Add a route on edge-01, commit it,
and deliberately do not save.
configure
set protocols static route 192.0.2.128/25 next-hop 203.0.113.2
set protocols static route 192.0.2.128/25 description 'committed, not saved'
commit
exit
The route is live. show ip route finds it, the FIB has it, traffic would use
it. Now ask the only question that matters:
[edit] vyos@edge-01# compare saved[edit]
+protocols {
+ static {
+ route 192.0.2.128/25 {
+ next-hop 203.0.113.2 {
+ }
+ }
+ }
+}Illustrative output
Any output at all from compare saved means the running configuration and the
boot file disagree, and the boot file wins at the next reboot. Reboot edge-01
and confirm the route is gone. Two minutes of your time, and it is the failure
mode Part LV-01 opens with.
The second lie: save <path> is not save. Re-apply the route, then take
what looks like a backup:
configure
set protocols static route 192.0.2.128/25 next-hop 203.0.113.2
commit
save /config/backups/task2.boot
exit
The directory has to exist first — mkdir -p /config/backups from the
operational shell — and the command reports success in the same shape as a
plain save, which is the trap.
[edit] vyos@edge-01# compare saved[edit]
+protocols {
+ static {
+ route 192.0.2.128/25 {
+ next-hop 203.0.113.2 {
+ }
+ }
+ }
+}Illustrative output
Identical output to the first case, from a completely different mistake. An
operator who has typed save once that day and seen “Done” will not type it
again, and the router is one unplanned reboot away from reverting a change
everybody believes is in place.
Now save properly, and confirm with the check that understands both syntaxes:
configure
save
compare saved
exit
Empty output is the proof. Record in the journal that compare saved is the
verification and a file timestamp is not.
Task 3 — Four backups, four different failures
You now have four artefacts, or you will by the end of this task. Produce each one, and write next to it in the journal the failure it survives and the failure it does not.
The boot file. /config/config.boot, written by save. Survives a reboot.
Does not survive a disk failure, a reinstall, or delete system image.
The commit archive. Written automatically on every successful commit.
$ show system commit0 2026-08-19 11:04:22 by vyos via cli
1 2026-08-19 10:58:07 by vyos via cli
2 2026-08-19 10:41:55 by vyos via cli
3 2026-08-19 10:39:12 by vyos via cliIllustrative output
Revision 0 is the running configuration and the numbers count backwards in time. The archive is as fresh as the last commit, which makes it the best source you have — and it lives on the router’s own disk, which makes it worthless in exactly the scenario the rest of this lab is about. Retention defaults to 20 entries, so on a busy router it can be a single afternoon.
A labelled on-box snapshot. Independent of archive pruning, and it holds states the archive cannot — an uncommitted repair, or a state you want to compare against rather than run.
configure
save /config/backups/pre-lab22.boot
exit
The off-box copy. Push it from ops; do not ask the router to fetch it.
# On ops. Substitute your own values before running:
ROUTER=192.0.2.11
LABEL=edge-01
STAMP=$(date +%Y%m%d-%H%M%S)
mkdir -p /home/vyos/configs
scp "vyos@$ROUTER:/config/config.boot" "/home/vyos/configs/$LABEL-$STAMP.boot"
ls -la /home/vyos/configs/
The timestamp in the filename is doing the job a version-control repository
does in production. Part LV-02 is unambiguous that the off-box copy belongs in
Git and that a single overwritten file is not a history; the lab uses stamped
filenames because the point being made here is about lag, not about the
version-control tool. Write the lag next to each one: the boot file is as fresh
as your last save, the archive as fresh as your last commit, this copy as
fresh as the last time you ran that scp.
Task 4 — Restore is not rollback, and compare is the only record of what you discard
Now simulate the thing that makes a restore expensive: somebody else’s work.
Fifteen minutes after your snapshot, a colleague adds a firewall rule. Apply it as them:
configure
set firewall ipv4 forward filter rule 20 action accept
set firewall ipv4 forward filter rule 20 description 'CHG-4471 monitoring probe to LAN'
set firewall ipv4 forward filter rule 20 inbound-interface name eth1
set firewall ipv4 forward filter rule 20 outbound-interface name eth2
set firewall ipv4 forward filter rule 20 protocol icmp
set firewall ipv4 forward filter rule 20 state new
commit
save
exit
Then break something yourself, in a way that looks like a bad change rather than a typo:
configure
delete protocols ospf area 0 network 198.51.100.0/24
commit
save
exit
The LAN prefix stops being advertised. From ops, the far service is still
reachable — the outbound path is fine — but the return path has no route, so
the ping stops. That asymmetry is worth noticing; it is the subject of Lab 24.
Before reaching for a file, ask the question Part LV-03 and the restore runbook both put first:
show system commit
The good state is revision 1. This is a rollback 1, not a restore, and doing
it as a restore is the expensive way to get the same result. Do it as a restore
anyway, this once, because the diff is the lesson.
configure
load /config/backups/pre-lab22.boot
compare
Read the whole diff, not the part you expected.
$ compare[edit firewall ipv4 forward filter]
-rule 20 {
- action accept
- description "CHG-4471 monitoring probe to LAN"
- inbound-interface {
- name eth1
- }
- outbound-interface {
- name eth2
- }
- protocol icmp
- state new
-}
[edit protocols ospf area 0]
+network 198.51.100.0/24Illustrative output
Two hunks, two completely different meanings. The + line is the restore doing
its job. The - block is a change somebody else made, for a ticket you have
never read, which this commit will silently delete. Nothing else in this lab —
and nothing in the router’s own output afterwards — will ever mention it again.
The commit produces one archive entry and no summary of what it reverted.
discard here, then take the cheap path:
discard
rollback 1
compare
commit
save
exit
Paste the compare output into the journal and annotate every line as either
“the restore” or “work being discarded”. That annotated diff is the change
record for a restore, and in production it is the artefact that goes in the
ticket.
Task 5 — The same-appliance restore, under a timer, verified in dependency order
Now do the restore properly, on a state the archive cannot reach. First take a fresh off-box copy, so the source is the state you validated at the end of Task 4 rather than the Task 3 one — otherwise this restore discards the colleague’s rule a second time and you will be reading a diff about that instead of about the fault:
# On ops. Substitute your own values before running:
ROUTER=192.0.2.11
STAMP=$(date +%Y%m%d-%H%M%S)
scp "vyos@$ROUTER:/config/config.boot" "/home/vyos/configs/edge-01-$STAMP.boot"
Now wreck edge-01 the way a bad change does — several nodes at once — and
recover from that copy rather than from anything on the box.
configure
delete interfaces ethernet eth2 address
delete protocols ospf
delete firewall ipv4 forward filter rule 10
commit
save
exit
That is a router with a LAN interface holding no address, no routing protocol and a forward filter that drops everything. It is up, reachable on MGMT, and completely out of service — which is the realistic shape of a bad change.
Push the off-box copy back and load it:
# On ops. Substitute the filename you just captured:
ROUTER=192.0.2.11
FILE=/home/vyos/configs/edge-01-20260819-110800.boot
scp "$FILE" "vyos@$ROUTER:/tmp/restore.boot"
ssh "vyos@$ROUTER" 'ls -la /tmp/restore.boot'
configure
load /tmp/restore.boot
compare
commit-confirm 15
exit
commit-confirm is not a formality here. A full restore replaces the interface
addressing, the management binding and the firewall in one commit, and any of
those can carry the session you are typing into. The timer returns the router
to a reachable state without requiring you to still be connected to it.
Size the timer for the verification you intend to run, not for how long the
commit takes. An OSPF adjacency takes tens of seconds to reach Full; a timer
that fires while you are still watching it revert a restore that was working.
Verify from operational mode, in dependency order. Order matters more than the individual commands:
show interfaces
show ip route
show ip ospf neighbor
show firewall ipv4 forward filter
Interfaces before addresses, addresses before adjacencies, adjacencies before
routes, routes before traffic. Out of order, you get confident wrong answers: an
adjacency stuck in Init because an interface never came up looks identical to
one stuck because the network statement is missing, and only the ordering
separates them.
Then the test that is not on the router at all:
$ ping 203.0.113.200 count 3PING 203.0.113.200 (203.0.113.200) 56(84) bytes of data.
64 bytes from 203.0.113.200: icmp_seq=1 ttl=63 time=0.95 ms
64 bytes from 203.0.113.200: icmp_seq=2 ttl=63 time=0.87 ms
64 bytes from 203.0.113.200: icmp_seq=3 ttl=63 time=0.90 msIllustrative output
Only now keep it and persist it:
configure
confirm
save
compare saved
exit
Task 6 — The replacement chassis: hw-id pins a MAC that does not exist here
Everything so far has been the same appliance. Now pin the interface names, the way a production configuration for real hardware does, and take the file next door.
On edge-01, using the MAC addresses you measured in Task 1:
configure
set interfaces ethernet eth0 hw-id 52:54:00:11:aa:01
set interfaces ethernet eth1 hw-id 52:54:00:11:aa:02
set interfaces ethernet eth2 hw-id 52:54:00:11:aa:03
compare
commit
save
exit
Substitute your own measured values — those three are the shape of a QEMU MAC,
not yours. hw-id binds a name to a specific NIC so that eth1 is always the
NIC in a known socket regardless of the order the kernel enumerated them. It is
the correct thing to have on a production router, and it is the thing that
makes the file non-portable.
Take a fresh off-box copy, then load it onto edge-02:
# On ops.
ROUTER=192.0.2.11
STAMP=$(date +%Y%m%d-%H%M%S)
scp "vyos@$ROUTER:/config/config.boot" "/home/vyos/configs/edge-01-$STAMP.boot"
scp "/home/vyos/configs/edge-01-$STAMP.boot" "vyos@192.0.2.12:/tmp/restore.boot"
Before loading, do what Part LV-04 puts first and the incident always puts last: compare what the file expects against what the box has.
show interfaces
show configuration commands | match "hw-id"
Now load and commit, and let it fail:
configure
load /tmp/restore.boot
compare
commit
$ commit[ interfaces ethernet eth1 ]
Error: Interface "eth1" hardware ID does not match any interface on this system
[[interfaces ethernet eth1]] failed
Commit failedIllustrative output
Read what just happened carefully, because the good news is easy to miss. The
commit engine is transactional with respect to the running configuration: the
validator rejected the load and edge-02 is exactly as it was. A failed commit
is not an outage.
The candidate, however, still holds the rejected content. The recovery is to
fix the offending nodes or discard — not to retry the same commit.
Fix the hardware-bound nodes in the candidate, before committing, using
edge-02’s own row of the identity table:
delete interfaces ethernet eth0 hw-id
delete interfaces ethernet eth1 hw-id
delete interfaces ethernet eth2 hw-id
set system host-name edge-02
set interfaces ethernet eth0 address 192.0.2.12/24
delete interfaces ethernet eth0 address 192.0.2.11/24
set protocols ospf parameters router-id 192.0.2.12
compare
commit
save
exit
Three separate corrections in that block, and only the first one had an error message.
The address change is the second failure mode from Part LV-04, and it is the
one that reaches production. The restored file carried 192.0.2.11 — the
address edge-01 is still using, because edge-01 is still powered on. Commit
it unchanged and two routers claim one address: ARP flapping on the MGMT
segment, and a management path that works intermittently for reasons that look
like anything except a restore.
The router-id change is the third. It came in from the file too, and two OSPF routers sharing a router-id is its own class of incident.
Task 7 — The restore that commits cleanly and routes nothing
Delete the hw-id lines and the commit succeeds. That is what makes this the
most dangerous shape a restore can take, and it is exactly the shortcut the
break/fix scenario vyos-bf-restore-wrong-interface is built on.
You have already deleted them. Now finish the job the way a rushed engineer
would: move edge-02 into edge-01’s place.
# On the hypervisor. Substitute your own VM and bridge names before running:
VM_OLD=edge-01
VM_NEW=edge-02
virsh destroy "$VM_OLD"
# Attach VM_NEW's vNICs to MGMT, LAN and TRANSIT -- in the order the
# chassis presents them, which for this VM puts LAN on eth1.
edge-02 now has the restored configuration and is cabled by socket label
rather than by bus position. Commit and look at it:
show interfaces
show ip ospf neighbor
Everything is green. Three interfaces up, three addresses, one OSPF adjacency
in Full. There is no error anywhere on the box. And from ops:
$ ping 203.0.113.200 count 3PING 203.0.113.200 (203.0.113.200) 56(84) bytes of data.
From 198.51.100.20 icmp_seq=1 Destination Host Unreachable
From 198.51.100.20 icmp_seq=2 Destination Host Unreachable
--- 203.0.113.200 ping statistics ---
3 packets transmitted, 0 received, +2 errors, 100% packet lossIllustrative output
Now find it. The configuration is correct, so no configuration command will show you anything wrong. The evidence is one layer down: which MAC is currently answering to each name.
# On edge-02, operational mode.
for i in eth0 eth1 eth2; do
printf '%-6s %s\n' "$i" "$(cat /sys/class/net/$i/address)"
done
# And compare against the row you wrote in Task 1.
cat "$HOME/lab22/chassis-identity.txt"
eth1 holds the address 203.0.113.1/30 and is attached to the LAN bridge;
eth2 holds 198.51.100.1/24 and is attached to TRANSIT. The adjacency formed
because peer is on the segment eth2 is now cabled to, and OSPF does not care
which interface name it runs over. Every single indicator is consistent with a
working router.
There are two fixes and they are not equivalent.
Re-cable. Move the vNICs so the names land where the configuration expects. Immediate, no reboot, and leaves the configuration byte-identical to the off-box copy. It also makes this the one chassis in the estate whose socket labels disagree with the standard, and hands the same confusion to the next person in reverse.
Re-derive hw-id from this chassis. Set each name to the MAC that is in
the socket you want it to be, from the measured table:
configure
set interfaces ethernet eth1 hw-id 52:54:00:22:bb:03
set interfaces ethernet eth2 hw-id 52:54:00:22:bb:02
compare
commit
save
exit
This keeps the cabling standard true for everyone and puts the chassis-specific data back where the design always had it. It needs a reboot for the renaming to settle, and it means the off-box copy for this router now differs from the one you restored — which has to be pushed back, or the next backup records it as an unexplained rewrite.
Choose one deliberately and write down why in the journal. “Deleted the hw-id
lines to get a clean commit” is the sentence that appears in the break/fix
ticket for this exact failure, and it is a decision made by nobody.
Task 8 — Time the drill
Everything above was untimed. Do it once more against a clock, because Part LV-06 is clear that a recovery procedure with no measured time is a hypothesis.
Rebuild edge-02 from the ISO — a genuinely bare chassis, empty commit archive,
nothing on disk — and recover it from the off-box copy alone. Record each stage:
| Stage | Your time | Part LV-06 reference |
|---|---|---|
| Confirm the failure (console, not ping) | 5 min | |
| Obtain the replacement (here: install from ISO) | 60 min | |
| Transfer and load the configuration | 15 min | |
| Adapt the hardware-bound nodes | not in the table | |
| Adjacency verification | 10 min | |
Traffic verification, from ops | 10 min | |
| Total | 130 min |
Two things to notice in your own numbers.
The reference table has no row for adapting the hardware-bound nodes, and in this lab that stage is where you lost the most time — twice, once to an error message and once to no error message at all. A stage table that does not name it is a stage table written by somebody who has only ever restored onto the same appliance.
And the bottleneck will not be the restore. It is the install, exactly as Part LV-06 predicts, which is why the recommendation is a pre-configured spare rather than a faster recovery procedure. Write down which stage you would actually attack, and what it would cost.
Validation
- The chassis identity table exists for both routers, with interface name, measured MAC, and the bridge each vNIC is attached to.
- In Task 2,
compare savedproduced output in both cases and empty output after a plainsave, and the reboot removed the unsaved route. - Four backup artefacts exist, each labelled with the moment it was taken and the failure it survives.
- The Task 4
compareoutput is in the journal, annotated line by line as either the restore or discarded work, and it names the colleague’s rule. - The Task 5 restore was committed with
commit-confirm, verified in dependency order, confirmed, saved, andcompare savedafterwards returned nothing. - The Task 6 commit was refused by the validator with a hardware-ID error, and
show interfacesonedge-02was unchanged after the failed commit. - After the Task 6 corrections,
edge-02holds192.0.2.12and a router-id of192.0.2.12, and no address appears twice in the estate. - In Task 7,
show interfacesandshow ip ospf neighboronedge-02both reported a healthy router whilepingfromopsfailed, and the measured MAC table is what identified the cause. - A stage-by-stage recovery time exists for Task 8, with a named bottleneck.
Expected Outcome
A two-chassis lab in which you have produced every backup a VyOS router produces and made each of them fail at something specific: the boot file at a reinstall, the archive at a dead disk, the on-box snapshot at a chassis replacement, and the off-box copy at nothing except its own lag — which is the reason it is the one that matters.
More usefully, you have seen the two shapes a cross-appliance restore takes. One of them is loud: the validator refuses the commit, names the interface, and leaves the running configuration untouched. The other is silent: it commits, it comes up, every operational command agrees the router is healthy, and no traffic passes. You have a written procedure that catches the second one, and it is a table of MAC addresses.
Troubleshooting
save /config/backups/pre-lab22.boot reports that the directory does not
exist. It is not created for you. mkdir -p /config/backups from the
operational shell, once per router. /config is on the persistent partition, so
it survives a reboot — but not a reinstall.
compare saved prints the whole configuration rather than a diff. You are
comparing against an empty or absent boot file, which on a freshly installed
router is the correct answer. Run a plain save and try again.
The Task 6 commit succeeds when you expected it to fail. The hw-id values
in the file happen to match edge-02, which means you captured the file before
the Task 6 set interfaces ... hw-id commit, or you measured edge-01’s MACs
on the wrong VM. Check with
show configuration commands | match "hw-id" on the candidate.
The OSPF adjacency will not form after a restore. Work outward and in
order. show interfaces first — an adjacency cannot form over an interface
that has no address. Then check that both ends agree on the network statement
covering the transit prefix, and that the router-ids are distinct.
show ip ospf interface names the area and the timers each side is using.
ops can reach 203.0.113.200 but the ping from edge-01 itself was
already working. That is the normal state and it is why the test is run from
ops. A router reaching its own neighbour proves the local interface works; it
proves nothing about forwarding, the firewall, or the return path.
The restore committed and then reverted itself a few minutes later. The
commit-confirm timer expired before you ran confirm. Reload and use a timer
sized for adjacency formation rather than for the commit.
Two routers are fighting over 192.0.2.11 on the MGMT segment. Task 6’s
address correction was skipped, or edge-01 was never powered off. Power it
off, then clear the neighbour entry on the other hosts with
ip neighbour flush dev eth0 before deciding whether the problem is fixed.
Everything is up on edge-02 and nothing routes, and the MAC table matches.
Then the cabling is right and something else is wrong. Re-check the forward
filter: rule 10 names inbound-interface eth2 and outbound-interface eth1,
and if you fixed the naming by re-cabling rather than by hw-id, those two are
now reversed. Read the rule’s packet counter in
show firewall ipv4 forward filter.
Cleanup
Everything here is on isolated bridges, so cleanup means restoring a known state rather than restoring service.
Step 1. If you are keeping the topology for the backup and restore break/fix
scenarios, stop here and save on each router. Note in the journal which
chassis is currently in service, whether its interface names are pinned with
hw-id, and which bridge each vNIC is on — the same table Task 1 produced. The
next person to open this lab needs it for the same reason you did.
Step 2. To return the estate to its Task 1 state, power off edge-02, restore
edge-01 from the on-box snapshot, and remove the lab-only nodes:
configure
load /config/backups/pre-lab22.boot
compare
commit
save
exit
Step 3. Remove the artefacts the lab left behind on edge-01:
# On edge-01, operational mode.
ls -la /config/backups/
rm -f /config/backups/task2.boot
rm -f /tmp/restore.boot
Leave /config/backups/pre-lab22.boot until you have confirmed the router is
where you want it. It is the only thing that gets you back.
Step 4. On ops, decide what happens to the off-box copies. They contain the
full configuration of both routers including any credentials, which in this lab
is a lab password and in production is not:
# On ops.
ls -la /home/vyos/configs/
Step 5. To remove the topology entirely, delete the four VMs and the four
bridges on the hypervisor. Confirm with ip -brief link show on the host that
no lab bridge survives; an orphaned bridge with no members is the kind of thing
somebody reuses six months later assuming it means something.
What You Learned
- “The backup” is four different artefacts with four different lags. The
boot file is as fresh as your last
save, the archive as fresh as your last commit, the snapshot as fresh as whenever somebody typed it, and the off-box copy as fresh as its schedule. Picking one is choosing how much recent work to discard, and nothing in the restore output will tell you what that was. compare savedis the verification; a timestamp is not. Two entirely different mistakes — a missingsave, and asave <path>that did not touch the boot file — produce the same symptom and are both invisible until a reboot.comparebefore a restore is the change record. It is the only place the work you are about to delete becomes visible, and it deserves to be pasted into the ticket rather than skimmed.- A failed commit is not an outage. The validator refusing a load leaves the running configuration untouched. The candidate keeps the rejected content, so the recovery is to fix or discard — never to retry the same commit and hope.
- A cross-appliance restore is a migration. Interface names,
hw-id, addressing and router-id are all bound to the chassis the file was written for, and only the first of them produces an error message. Fix them in the candidate, before the commit. - The dangerous restore is the one that commits cleanly. Correct names,
correct addresses, adjacency
Full, no traffic. Nothing the router can observe is wrong, because the fault is in the mapping between the configuration and the room. - The cheapest artefact in a disaster-recovery plan is a table of MAC addresses. Three lines per chassis, written while everything works. It is the difference between a fifteen-minute recovery and an afternoon.