Final assessment · expert · ~180 min · pass ≥ 80%

Backup, Restore & Disaster Recovery for Production Infrastructure — Final Practical Assessment

Objectives

Take ownership of an undocumented backup estate and produce an inventory that supports recovery rather than describing configuration
Separate the claim a piece of evidence actually supports from the claim it is being used to support
State the data-loss window an architecture permits rather than the one a schedule implies
Argue RPO and RTO consequences separately, and name which stage of the recovery consumes the budget
Scope blast radius by what one credential, one administrator or one propagated operation can reach
Sequence remediation so that a weak protection is never removed before its replacement has been proven
Validate a recovery from an invariant recorded before the incident rather than from the absence of errors
Propose structural prevention that retires a failure class instead of an instruction that reduces its likelihood

12 graded questions · last verified 2026-08-28

Assessment questions

Objective questions are scored automatically. Scenario answers use a second-stage rubric for evidence, remediation, and rollback.

Knowledge check · 12 questions

Q1. Finding 1. Produce evidence, failure coverage gap, data-loss risk, RPO impact, RTO impact, security impact, blast radius, remediation, restore validation and prevention for the verification defect.

The dashboard shows "backup health: 99.8%" and has done for two years. The figure is derived from the exit status of the client timers. A separate nightly structural integrity check runs against every repository and has also reported success on every run for two years; it reads the repository indexes and trees and does not read the stored data. No restore has ever been performed from these repositories. The recovery objectives circulated to the business were estimated from how long the nightly backups take to run.
Q2. Finding 2. Produce evidence, failure coverage gap, data-loss risk, RPO impact, RTO impact, security impact, blast radius, remediation, restore validation and prevention for the snapshot defect.

Forty-one virtual machines run on a three-host cluster backed by one shared storage array. The array takes hourly snapshots of every datastore and retains them for fourteen days. The handover describes this as "hourly backups, two weeks of retention", and that retention figure appears in the recovery objectives circulated to the business. Nothing else protects the virtual machine estate.
Q3. Finding 3. Produce evidence, failure coverage gap, data-loss risk, RPO impact, RTO impact, security impact, blast radius, remediation, restore validation and prevention for the file server defect.

A file server holds 3.1 TB of departmental shares and is copied every night to a second server in the same rack. The destination is overwritten in place, so the copy always reproduces the current state of the source, including deletions. The handover describes this as "a nightly full backup, kept on separate hardware". Three weeks ago a folder of contracts went missing and was noticed four days later. The second server had the same folder missing. The ticket was closed as "data confirmed deleted at source, not recoverable".
Q4. Finding 4. Produce evidence, failure coverage gap, data-loss risk, RPO impact, RTO impact, security impact, blast radius, remediation, restore validation and prevention for the replica defect.

Two PostgreSQL instances carry the order and billing systems. The primary streams to a hot standby in the second rack. The handover lists the standby under "backup and recovery" and no other mechanism is listed for these two databases. Failover has been tested twice and worked both times. Write-ahead log archiving is configured on the primary. The archive target is a directory on the primary itself. There is no monitoring on the archiver and no record of anyone having replayed the archive.
Q5. Finding 5. Produce evidence, failure coverage gap, data-loss risk, RPO impact, RTO impact, security impact, blast radius, remediation, restore validation and prevention for the warehouse consistency defect.

A reporting warehouse is protected by a nightly script that stops nothing, copies the entire data directory with a file-level copy while the service is running, and tars the result. The script has run every night for four years. It was tested once, in 2023, by starting a copy on a spare host. The instance came up, completed recovery, and returned the expected row count. The test was recorded as passed.
Q6. Finding 6. Produce evidence, failure coverage gap, data-loss risk, RPO impact, RTO impact, security impact, blast radius, remediation, restore validation and prevention for the credential defect.

Every client authenticates to the NAS with the same account, which has full read, write and delete rights over every repository. The account credential is present on every client so the timers can run unattended. The morning synchronisation job that copies the NAS repositories to object storage uses a second credential, held on the NAS, which has full rights over the bucket.
Q7. Finding 7. Produce evidence, failure coverage gap, data-loss risk, RPO impact, RTO impact, security impact, blast radius, remediation, restore validation and prevention for the key custody defect.

The repositories are encrypted. The passphrase is held in a file on each client so the timer can run unattended, and a copy is recorded on a wiki page. The wiki runs on one of the virtual machines in the cluster protected by the same repositories. No other copy of the passphrase is known to exist. Nobody has attempted to open a repository using only material held outside the primary site.
Q8. Finding 8. Produce evidence, failure coverage gap, data-loss risk, RPO impact, RTO impact, security impact, blast radius, remediation, restore validation and prevention for the object storage retention defect.

The NAS repositories are copied to an S3-compatible bucket each morning by a job that mirrors the current state of the NAS, including deletions. Bucket versioning was enabled two years ago. Object locking was never enabled: the attempt against the existing bucket was refused and the ticket was closed as "not supported". A lifecycle rule expires non-current object versions after seven days. It was added during a budget review and is not mentioned in the handover.
Q9. Finding 9. Produce evidence, failure coverage gap, data-loss risk, RPO impact, RTO impact, security impact, blast radius, remediation, restore validation and prevention for the recovery time defect.

A copy of the bucket is transitioned to a deep archive storage class after thirty days and the handover names this as the disaster recovery copy. The published recovery time objective for the estate is four hours. The archive class documents a retrieval latency measured in hours before any bytes can be transferred. No retrieval has ever been performed. Every duration in the recovery objectives was estimated from how long the backups take to run.
Q10. Finding 10. Produce evidence, failure coverage gap, data-loss risk, RPO impact, RTO impact, security impact, blast radius, remediation, restore validation and prevention for the Kubernetes defect.

Every manifest lives in a Git repository and is reconciled continuously. The handover states that the Git repository is the Kubernetes backup and that the cluster can be rebuilt from it in twenty minutes. That claim has been demonstrated once, on an empty cluster. Six applications use persistent volume claims backed by the same storage array as the virtual machines. Nothing copies those volumes anywhere, and two of them hold the only copy of data the applications produce. A control-plane snapshot is taken every six hours by a timer on the first control-plane node and written to that node's local disk. Only the most recent snapshot is retained; the previous one is deleted at the start of each run.
Q11. Finding 11. Produce evidence, failure coverage gap, data-loss risk, RPO impact, RTO impact, security impact, blast radius, remediation, restore validation and prevention for the disaster recovery site defect.

A second facility fifty kilometres away holds compute and storage sized for the tier-one systems and is described in the handover as "DR: solved". The site has no copy of the backup catalogue, no copy of the repository passphrase, and no independent path to the certificate authority key material, which lives in the secret manager in the primary site. The secret manager unseals using a certificate issued by that authority. Public DNS records for the customer-facing services have a time to live of 86400 seconds. There is no documented failback procedure. The only exercise ever performed brought up one application server in the second site, confirmed the login page rendered, and was recorded as a successful DR test.
Q12. Finding 12. Produce evidence, failure coverage gap, data-loss risk, RPO impact, RTO impact, security impact, blast radius, remediation, restore validation and prevention for the inventory defect. You are under pressure on this one.

There is no inventory listing, for each system, what protects it, where the copies are, who can delete them, and when a restore was last proven. The document lists mechanisms by tool rather than systems by protection, so a system with no protection does not appear as a gap; it simply does not appear. The director has read your first eleven findings. Their response is: "This is a lot. Pick the three that matter, fix those this quarter, and we will look at the rest next year. And do not write another document — the last engineer wrote documents and look where that got us."

Passing score: 80%. Answers are checked in this browser.

Backup, Restore & Disaster Recovery — Final Practical Assessment

This assessment gives you an estate rather than a question bank. It is small, ordinary, and entirely plausible: a virtualisation cluster, a file server, three databases, a deduplicating backup platform, an object storage bucket, a Kubernetes cluster and a second site. It contains twelve defects. None of them is a trick, none of them is the result of incompetence, and every one of them exists somewhere in production today.

You have three hours and you are not expected to change anything. What is being graded is whether you can establish what is true, decide what to do first, and defend the ordering.

How the twelve findings are presented

Each finding is a scenario carrying the part of the handover that matters for it. The description is factual and deliberately incomplete, in the same way a real handover is. Where a scenario states a belief held by a team, a figure circulated to the business, or an instruction given by a colleague, that belief, figure or instruction is part of the material and may be wrong.

Several findings describe mechanisms that are correct for the failure they address and are being asked to cover a failure they do not. An answer that condemns such a mechanism scores lower than one that credits what it does and names precisely what it does not.

What each answer must contain

Ten headings, in this order: Evidence, Failure coverage gap, Data-loss risk, RPO impact, RTO impact, Security impact, Blast radius, Remediation, Restore validation, Prevention. The full definition of each is in the sections above and it is worth reading before you start rather than after.

The weightings are Evidence 15%, Remediation 15%, and 10% each for Failure coverage gap, Data-loss risk, RPO impact, RTO impact, Security impact and Restore validation, with 5% each for Blast radius and Prevention. The pass mark is 80%, averaged across the twelve findings.

The ordering rule

No existing protection is removed, reconfigured or replaced until its successor has produced a copy that has been restored and verified. An answer that retires the current mechanism first scores zero for Remediation, however correct the destination architecture is, because the estate spent the intervening period with less protection than it started with.

The exception, and it is the only one, is a credential that can destroy backups. There, containment comes first: remove the delete capability or rotate the credential before anything else, including before you finish designing the replacement identity model. A live delete-capable credential on a production host is not a design problem for next sprint; it is the mechanism by which the entire estate is lost in a single evening.

Three habits that are graded throughout

Evidence from an independent channel. Where the obvious source of truth is the system under suspicion, name a different one. A green dashboard, a structural integrity check, a handover document, a passed test from 2023 and a file your own restore just wrote are all evidence that something happened. None of them is evidence that recovery works.

Precision about what is not affected. Several of these findings touch four of the ten headings and not the other six. An answer that inflates every heading for every finding is not a thorough answer; it is an answer that cannot tell the difference between an availability defect and a durability one, and it will be marked as such.

Structural prevention, named as such. Every finding will accept a procedural answer, and every procedural answer will be marked lower than the structural one that retires the failure class. Propose both if you like, but say which is which. This estate was built entirely from procedural controls by a capable engineer who was here for six years, and it reached this state anyway.