Can the attacker delete the backups?

Measuring one endpoint’s semantics instead of trusting a compatibility page settles a narrow question well, and it leaves the large one open. The large one is not whether your object store implements retention the way the documentation says. It is whether the person who has just taken your production estate can reach your recovery points and remove them. That question is the difference between a backup system that survives accidents and one that survives an adversary, and almost every estate answers it by assumption. It is answerable by measurement, and the measurement is cheap.

The credential you have to assume the attacker holds

Start with the threat model, because most backup designs are built against the wrong one. The implicit adversary in a lot of documentation is an outsider with no credentials, probing from the internet, stopped by a firewall and a bucket policy. That adversary exists and is largely uninteresting, because the estate already stops them.

The adversary that matters holds the credentials of a production administrator. This is not an exotic assumption; it is the ordinary outcome of a successful intrusion. Somebody phished a session, a build runner leaked an environment variable, an application server was compromised and the backup client’s configuration file was sitting on it in plain text. From that point on, every question about what the estate permits is a question about what those credentials permit, and every control that is enforced by policy attached to an identity is a control the intruder now administers.

So the honest form of the question is a walk across the estate, one hop at a time, asked about that specific credential rather than about the estate in general:

1. Can it reach the backup destination at all? Network path, endpoint resolvable, port open from a compromised production host.
2. Can it authenticate to the destination? Are the backup credentials on the host that was compromised, or somewhere the compromised host cannot read?
3. Can it delete objects or archives? The ordinary delete verb, the one the backup tool itself uses during retention.
4. Can it delete versions? On a versioned store this is the operation that actually destroys data, and it is a different permission from the previous one.
5. Can it change retention? Shortening a retain-until date destroys a recovery point later, quietly, without a delete appearing in any log.
6. Can it change lifecycle rules? A rule that expires non-current versions after one day is a deletion scheduled by configuration, not by a command.
7. Can it revoke the backup identity? Removing the writer stops new recovery points from appearing while the existing ones age out under retention.

Each of those is a separate answer, and the answers do not follow from each other. An estate can refuse question four and accept question five, in which case it is protected against an attacker who is in a hurry and not against one who is patient. The rest of this lesson is about producing those answers as evidence rather than as opinion.

Measured: what the production identity was able to do

The capture below builds the smallest honest version of this test. A bucket is created with locking enabled, a default COMPLIANCE retention of three days is applied, and a 38-byte backup object is written into it. Then a second identity called production is created and given the built-in readwrite policy — full read, write and delete on the bucket. The capture describes it exactly as it should be understood: the credentials a backup client runs under, and therefore the credentials an attacker holds after taking a production host.

The first destructive attempt is the one an attacker makes by reflex.

$ mc rm prod/rbdr-immutable/backup-0900.tar

  Created delete marker `prod/rbdr-immutable/backup-0900.tar` (versionId=4b3c593c-e8ad-444d-aa87-89e380a1fbae).
>>> exit code: 0

--- is the object still listed? ---

>>> exit code: 0

--- and with versions shown? ---
[2026-08-28 13:28:22 UTC]     0B STANDARD 4b3c593c-e8ad-444d-aa87-89e380a1fbae v2 DEL backup-0900.tar
[2026-08-28 13:28:20 UTC]    38B STANDARD 133fd99f-1f98-41c0-9d08-95e6e2944157 v1 PUT backup-0900.tar
>>> exit code: 0

Exit code 0. The object left the ordinary listing entirely — the second command returned nothing at all — while the versioned listing shows the 38-byte v1 PUT still stored underneath a zero-byte v2 DEL marker. So the answer to question three, for this identity on this bucket, is yes, and it succeeds. The answer to the question the operator actually cares about — was a recovery point destroyed — is no. Those two answers came out of one command, and only one of them appears in an exit code.

Now the same question against a repository rather than a bucket, because most estates have both and the mechanisms are not comparable. The capture initialised a Borg repository, wrote three archives into it, and then set the flag that almost everyone believes is the answer here: borg config /work/aorepo append_only 1, which the capture confirms as append_only = 1.

$ borg config /work/aorepo append_only 1

--- archives before the attack ---
day1                                 Fri, 2026-08-28 13:58:09 [edf6f20cb15b5a2ed56ed0a8f4abe6e02e4d6a1e305729d1febb01608482195f]
day2                                 Fri, 2026-08-28 13:58:09 [dc55669922a0cbf88e0c74390976f69dd69c7ae81f90d3ea22b977c4b24c35e7]
day3                                 Fri, 2026-08-28 13:58:09 [06305e70a6a1ea7dc8c56e68eae5290ad7c52a4f22b559e8c46ac03b6398ad6a]

THE COMMON BELIEF: append-only makes delete fail. Test it.
$ borg delete /work/aorepo::day1

>>> exit code: 0
$ borg delete /work/aorepo::day2
>>> exit code: 0
$ borg delete /work/aorepo::day3
>>> exit code: 0

--- what the attacker now sees ---
(empty)

Three deletions, three exit codes of 0, and an empty archive listing. The protection everybody names when asked how their Borg repository resists an attacker did not refuse a single one of them. The same capture, against a repository with ordinary access and no flag at all, showed borg delete /work/repo::day1 returning exit code 0 and the listing coming back with only day2 on it — an attacker holding the backup credentials removed a recovery point and the repository cooperated, because ordinary repository access includes delete.

What append-only did do is visible one line further down: after the three deletions the capture recorded repository data still occupying: 41M under the repository’s data directory. The segments were never freed. Append-only forbids compaction, so the deletes were reversible by rolling the repository back to its last good transaction — which the capture then did, recovering all three archives and extracting day1 successfully. That recovery required filesystem access to the repository server, which is precisely the access the compromised client does not have, and it was possible only because borg compact had not yet run.

The same estate, with a retention no principal could lift

The bucket in the first capture was not defenceless; the delete marker simply was not the operation the defence was aimed at. The next attempts name the version id, which is the request that actually destroys stored data, and they escalate the identity making it.

$ mc rm --versions --version-id 133fd99f-1f98-41c0-9d08-95e6e2944157 prod/rbdr-immutable/backup-0900.tar

  mc: <ERROR> Failed to remove `prod/rbdr-immutable/backup-0900.tar`. Object, 'backup-0900.tar (Version ID=133fd99f-1f98-41c0-9d08-95e6e2944157)' is WORM protected and cannot be overwritten
>>> exit code: 1

--- can the production identity force it? ---
$ mc rm --bypass --version-id ... prod/rbdr-immutable/backup-0900.tar
mc: <ERROR> Failed to remove `prod/rbdr-immutable/backup-0900.tar`. Object, 'backup-0900.tar (Version ID=133fd99f-1f98-41c0-9d08-95e6e2944157)' is WORM protected and cannot be overwritten
>>> exit code: 1

--- can the FULL ADMIN force it? ---
mc: <ERROR> Failed to remove `lab/rbdr-immutable/backup-0900.tar`. Object, 'backup-0900.tar (Version ID=133fd99f-1f98-41c0-9d08-95e6e2944157)' is WORM protected and cannot be overwritten
>>> exit code: 1

--- final state of the backup object ---
[2026-08-28 13:28:22 UTC]     0B STANDARD 4b3c593c-e8ad-444d-aa87-89e380a1fbae v2 DEL backup-0900.tar
[2026-08-28 13:28:20 UTC]    38B STANDARD 133fd99f-1f98-41c0-9d08-95e6e2944157 v1 PUT backup-0900.tar
>>> exit code: 0

Three refusals at exit code 1, and the third one carries the whole argument. The production identity could not remove the version. The production identity could not force it with --bypass. And the administrator alias that created the bucket, created the users and wrote the retention policy got the identical refusal. There is no role above this. The final listing shows both versions still present, the 38-byte original intact beneath the marker the attacker placed on it twenty-two minutes past the hour.

Note that this outcome is a property of the retention mode rather than of object lock as a feature. The same capture, under a GOVERNANCE retention with the same three-day period, saw the administrator’s --bypass return Removed ... at exit code 0. An estate that reports “object lock is enabled” has told you which feature is present and not which of these two results it produces.

The audit: every identity, every destructive operation, one exit code

The method follows directly from the walk. Build the list of identities that can reach the backup destination — not the list from the design document, the list from the destination’s own user and policy inventory, plus anything holding a credential that can create users. For each one, attempt every destructive operation against a test object in a test bucket or a scratch repository, and write down the exit code and the message.

ALIAS=prod
BUCKET=rbdr-audit
OBJECT=probe.tar
TARGET="$ALIAS/$BUCKET/$OBJECT"
VERSION_ID=133fd99f-1f98-41c0-9d08-95e6e2944157

mc ls --versions "$TARGET"

mc rm "$TARGET"
echo "plain delete exit: $?"

mc rm --versions --version-id "$VERSION_ID" "$TARGET"
echo "version delete exit: $?"

mc rm --bypass --version-id "$VERSION_ID" "$TARGET"
echo "bypass delete exit: $?"

mc retention set --default GOVERNANCE 1d "$ALIAS/$BUCKET"
echo "retention change exit: $?"

Run that with each identity’s credentials in turn, and the output is the audit. For the estate measured above, the completed table looks like this, and it is the only honest answer to the question in the title:

Two of those rows would have been predicted wrongly by almost anyone reading the configuration. The first row looks like a successful destruction and is not; the last row looks like a refusal and is not. That is the entire justification for running the probe rather than reasoning about the policy: the configuration describes intent, and the exit code describes behaviour.

The rows the table still needs

Deletion is the loudest destructive operation, not the only one, and an audit that stops at rm reports a security posture the estate does not have.

Retention change belongs on the list because it destroys recovery points on a delay. An identity permitted to shorten a retain-until date, or to move a bucket from compliance to governance for newly written objects, has arranged for tomorrow’s deletions to succeed without issuing one today. Lifecycle rules belong on it for the same reason: a rule that expires non-current versions after one day removes exactly the copies that a delete marker leaves behind, and it does so as a scheduled background action that no interactive command log will show. Both of these are configuration writes, so they are questions about who can call the configuration APIs, not about who holds delete.

Revoking the backup identity belongs on the list because it attacks the future rather than the past. An attacker who cannot destroy the stored versions can still remove the writer, at which point recovery points stop appearing while the existing ones age towards their expiry. The estate keeps every guarantee it made about the copies it has and quietly stops making new ones, and the only signal is an absence.

Finally, the table has an expiry date. It describes the estate on the day the probe ran. A new user, a broadened policy, a bucket created without the locking flag, a repository moved to a host the production network can reach — any of those changes an answer without changing the document that recorded it.

Production discipline

1. Assume the production credential is already the attacker’s. Design and audit against an adversary holding valid administrative credentials, because that is the ordinary outcome of an intrusion; an outsider with no credentials is a threat model the estate already handles.
2. Enumerate identities from the destination, not from the diagram. List every principal the backup store itself knows about, plus every principal that can create or re-policy one, before asking what any of them may do.
3. Attempt the operation instead of reading the policy. The measured readwrite identity returned exit code 0 on mc rm and exit code 1 on the version delete; no reading of the policy document produces both of those facts, and the append_only = 1 repository that emptied at exit code 0 shows what reading produces instead.
4. Record exit codes and messages, never adjectives. A row saying is WORM protected, exit 1 survives an argument. A row saying “protected” does not, and it does not distinguish the compliance result from the governance one, where the same administrator’s --bypass returned exit code 0.
5. Re-run the audit on a schedule and alert on divergence. The table describes the estate on the day it was produced; a new identity, a widened policy or a bucket created without locking silently changes an answer, and only a repeated probe notices.

Cross-course references

• Secrets, PKI & Certificate Management for Infrastructure Engineers — Part XIII (Dynamic Credentials and Workload Identity) attacks the premise this lesson starts from. The whole threat model here rests on a long-lived backup credential sitting on a host that can be compromised; a short-lived, workload-bound credential changes what the intruder inherits and for how long, so the credential lifetime and the retention mode are two halves of one decision about whether recovery points survive an intrusion.
• Kubernetes for Production Sysadmins — Part LIX (kubectl auth) teaches the identical audit method on a different control plane: kubectl auth can-i asks the API what a specific identity may actually do rather than inferring it from role definitions, which is exactly what the exit-code table above is doing for a backup destination. If you already trust that method for cluster RBAC, the case for applying it to the store holding your recovery points is the same one.
• Observability for Production Sysadmins — Part LXIII (Synthetic Monitoring) supplies the mechanism that turns this lesson’s one-off measurement into a standing guarantee. A synthetic probe that periodically attempts the destructive operations and asserts the recorded exit codes is the only way the table stays true after the next policy change, and it fails loudly on the day somebody creates a bucket without the locking flag.

Quiz