Skip to main content
RunBook Academy

LinuxXLIX · RestorePartial restore

Partial and point-in-time restores - one file, one directory, one moment

Advanced⏱ ~18 minbashtarrsyncdiff

What you'll learn

  • Restore to a staging path and reconcile by diff rather than overwriting live data
  • Browse a backup repository without extracting it
  • Find the last good version of a file rather than the most recent one
  • Explain how a PostgreSQL point-in-time recovery is driven and where it must not be run
  • Recognise the partial-restore failures that quietly revert other changes

Prerequisites

Verified against Ubuntu 24.04 LTS · Debian 12 (Bookworm) · RHEL 9.x · Rocky Linux 9.x · AlmaLinux 9.x · Linux kernel 6.1 LTS / 6.6 LTS · systemd 255+ · OpenSSH 8.7p1 (RHEL 9) / 9.6p1 (Ubuntu 24.04) · nftables 1.0.x · chrony 4.x · Pacemaker 2.1.x · Corosync 3.1.x · 2026-08-11

Not yet marked complete on this device.

The full-system restore is the one that gets rehearsed. The partial restore is the one that actually happens: somebody deleted a config file, a deployment overwrote a template, a table was truncated by a migration that was supposed to be read-only.

It has a completely different risk profile. The host is up. The service is running. Other people’s changes are live in the same directory you are about to write into. The failure mode is not “the restore did not work” - it is “the restore worked and took three weeks of unrelated changes with it”.

Rule one: never restore onto the live path

Configuration changestage first
$ sudo mkdir -p /var/tmp/restore-1842 && sudo chmod 700 /var/tmp/restore-1842

Extracting into a staging path costs one directory and buys three things: you can diff before committing, you can abandon the restore with no trace if it turns out to be the wrong version, and the live service keeps running on the files it already has while you decide.

Read-only / Safethe review step that saves the restore
$ sudo diff -ru /etc/nginx /var/tmp/restore-1842/etc/nginx
diff -ru /etc/nginx/nginx.conf /var/tmp/restore-1842/etc/nginx/nginx.conf
--- /etc/nginx/nginx.conf	2026-08-11 09:14:02.000000000 +0000
+++ /var/tmp/restore-1842/etc/nginx/nginx.conf	2026-07-21 02:00:11.000000000 +0000
@@ -18,7 +18,6 @@
worker_connections 4096;
-    multi_accept on;
}
Only in /etc/nginx/sites-enabled: reporting.conf

Illustrative output

Browsing a repository without extracting it

Finding the right version is most of the work, and every deduplicating backup tool can list and read individual paths without restoring anything.

Read-only / Safewhich archives exist
$ borg list /backup/repo
backup-2026-08-11T02:00:04  Tue, 2026-08-11 02:00:04 [3f9a...]
backup-2026-08-10T02:00:03  Mon, 2026-08-10 02:00:03 [7c2e...]
backup-2026-07-21T02:00:11  Tue, 2026-07-21 02:00:11 [b418...]

Illustrative output

Configuration changebrowse the archive as a filesystem
$ borg mount /backup/repo::backup-2026-07-21T02:00:11 /mnt/borg

Finding the last good version, not the last version

The newest backup contains the damage. That is normally the whole reason there is an incident: the file was corrupted or deleted at some point, and every backup since has faithfully captured that state.

  1. Establish when the damage happened, from the application log, the deployment record or the shell history - not from the backup timestamps.
  2. List the archives that bracket that moment. The last backup before it is the candidate.
  3. Verify the candidate before committing to it. Read the file out of the archive and check it is actually intact - a slow corruption may predate the incident you noticed.
  4. If the boundary is unclear, walk backwards. Compare consecutive archives and find where the file changes shape rather than content.
Read-only / Safewhat changed between two snapshots
$ restic -r /backup/repo diff 7c2e0000 3f9a0000 | grep nginx
M    /etc/nginx/nginx.conf
+    /etc/nginx/sites-enabled/reporting.conf

Illustrative output

Point-in-time recovery for databases

“Restore the file” has no meaning for a database. The unit is a consistent state at a moment, and reaching an arbitrary moment needs two ingredients: a base backup taken before it, and the write-ahead log covering the interval from the base backup to the target time.

PostgreSQL calls it PITR and drives it from configuration, not from a command:

# postgresql.conf on the RECOVERY instance - not the live one
restore_command = 'cp /backup/wal/%f %p'
recovery_target_time = '2026-08-11 09:12:00+00'
recovery_target_action = 'pause'
# the recovery.signal file is what puts the instance into
# targeted recovery; PostgreSQL 12 and later have no recovery.conf
sudo -u postgres touch /var/lib/postgresql/16/restore/recovery.signal

MySQL and MariaDB reach the same place from the other direction: restore the full backup, then replay the binary log up to the moment.

mysqlbinlog --start-datetime='2026-08-11 02:00:00' \
            --stop-datetime='2026-08-11 09:12:00' \
            /var/log/mysql/binlog.000147 > /var/tmp/restore-1842/replay.sql

Writing the replay to a file first, rather than piping it straight into mysql, is the same staging discipline as the file case. It is reviewable, it is re-runnable, and it does not apply anything until somebody has looked at it.

Knowledge check

Knowledge check · 4 questions

  1. Q1. You run `borg extract /backup/repo::backup-2026-07-21T02:00:11 /etc/nginx/nginx.conf` from your home directory. What happens?

  2. Q2. Which of these are correct handling for a single-file restore onto a live production host? Select all that apply.

  3. Q3. A PostgreSQL point-in-time recovery should be run against a separate data directory on a separate port, with recovery_target_action set to pause.

  4. Q4. A file was corrupted at some unknown point in the last month. Why is the newest backup usually the wrong one to restore from?

Passing score: 75%. Answers are checked in this browser.