Governance versus compliance retention: measure the difference

Objective

Lab 16 established what a COMPLIANCE retention refuses. This lab establishes what a GOVERNANCE retention does not refuse, using the same server, the same three-day period, the same object name and the same administrative credential.

You will create a second locked bucket, apply a GOVERNANCE default of three days, write a backup object, and restore it to prove a recovery point really existed. Then you will try to destroy the version twice: once plainly, once with --bypass. The two attempts return different exit codes, and that pair — set beside the pair Lab 16 produced — is the entire deliverable.

Architecture

One credential, three requests. The only variable is the retention mode on the bucket and the presence of one flag.

flowchart LR
    ADMIN["alias lab\nthe full administrator\nsame credential in every row"]
    ADMIN --> G1["mc rm --version-id\nbucket rbdr-governance\nGOVERNANCE 3DAYS"]
    ADMIN --> G2["mc rm --bypass --version-id\nbucket rbdr-governance\nGOVERNANCE 3DAYS"]
    ADMIN --> C1["mc rm --bypass --version-id\nbucket rbdr-immutable\nCOMPLIANCE 3DAYS, Lab 16"]
    G1 --> R1["is WORM protected\nexit 1\nversion survives"]
    G2 --> R2["Removed ...\nexit 0\nversion destroyed"]
    C1 --> R3["is WORM protected\nexit 1\nversion survives"]

Read the middle row against the bottom row. Same identity, same flag, same period, opposite outcomes — and the only difference is one word written into the bucket configuration before the first object arrived.

Requirements

• Mode B-nested. A disposable MinIO server in a container, an mc alias named lab holding administrative credentials, and nothing else. No second identity is needed; Lab 16 already measured the production one.
• Everything quoted below came from this pair of builds:

$ minio --version; mc --version

minio version RELEASE.2025-09-07T16-13-09Z (commit-id=07c3a429bfed433e49018cb0f78a52145d4bedeb)
Runtime: go1.24.6 linux/amd64
--- mc client version ---
mc version RELEASE.2025-08-13T08-35-41Z (commit-id=7394ce0dd2a80935aded936b09fa12cbb3cb8096)

• Lab 16’s bucket rbdr-immutable must still exist and still be inside its retain-until date, because Task 8 quotes its result and Cleanup depends on it being un-deletable. Every object this lab creates is prefixed rbdr-.
• The capture used the alias lab and the object name backup-0900.tar. Your version ids and timestamps will be your own; the messages and exit codes are the point, and those reproduce.
• What the capture covers, and what it does not. The transcript recorded Tasks 2, 3, 5, 6 and 8. It did not record the pre-lab inventory, the two restore attempts or the cleanup diff, so those embedded blocks retain their original provenance. A later complete run is captured in docs/courses/backup-dr/execution-evidence/backup-dr-lab-17-governance-versus-compliance-2026-08-29.txt and supports the last_executed date.

Scenario

A design review is deciding how the nightly backup bucket should be locked. Somebody proposes governance mode, on the reasonable grounds that it is reversible and the team is more likely to lock something by mistake than to be attacked. Somebody else says that reversibility is precisely what an intruder would use.

Both are right about what governance mode does. The review cannot settle the argument by reading the configuration, because the two modes look identical in a diff: GOVERNANCE 3DAYS against COMPLIANCE 3DAYS. Fifty minutes of running the deletion and writing down what came back settles it.

Tasks

Task 1 — Record the pre-lab state

Cleanup is diffed against this file, and one line of it matters more than the rest: the compliance-retained bucket rbdr-immutable is here now and must still be here at the end, because nothing in this lab can remove it before its retain-until date.

LAB="$HOME/rbdr-lab-17"
mkdir -p "$LAB"

{
  mc --version
  echo '--- rbdr- buckets present ---'
  mc ls lab/ | awk '{print $NF}' | sed -n 's#/$##p' | grep '^rbdr-' | sort
} | tee "$LAB/state.pre-lab"

Task 2 — Create the second locked bucket

Locking is a property the bucket is born with. Lab 16 measured what happens when it is requested afterwards, and the refusal is worth re-reading before you type the next command, because it is the reason --with-lock appears here rather than a retention set on an existing destination.

$ mc retention set --default COMPLIANCE 7d lab/rbdr-plain

   ...attempting to add locking to a bucket that was created without it:
mc: <ERROR> Unable to apply bucket lock configuration. Object Lock configuration cannot be enabled on existing buckets.
>>> exit code: 1

$ mc mb --with-lock lab/rbdr-immutable    (locking enabled at creation)
Bucket created successfully `lab/rbdr-immutable`.
>>> exit code: 0

mc mb --with-lock lab/rbdr-governance
echo "bucket create exit code: $?"

mc retention set --default GOVERNANCE 3d lab/rbdr-governance
echo "retention set exit code: $?"

mc version info lab/rbdr-governance

$ mc retention set --default GOVERNANCE 3d lab/rbdr-governance

  Object locking 'GOVERNANCE' is configured for 3DAYS.
>>> exit code: 0

Compare that line with the one Lab 16 produced. The subcommand, the period and the exit code are identical; a reviewer skimming two bucket configurations has almost nothing to notice.

Task 3 — Write the object and capture its version id

LAB="$HOME/rbdr-lab-17"
printf 'nightly application backup, taken at 09:00\n' > "$LAB/rbdr-b.tar"
sha256sum "$LAB/rbdr-b.tar" | tee "$LAB/sha256.0900"

mc cp "$LAB/rbdr-b.tar" lab/rbdr-governance/backup-0900.tar
echo "upload exit code: $?"

VID=$(mc ls --versions lab/rbdr-governance/backup-0900.tar | awk '{print $6}' | head -1)
echo "governance-protected version id: $VID" | tee "$LAB/result.txt"

The checksum is taken before the upload and stored outside the bucket, so the comparison in Task 4 is a real proof rather than the object agreeing with itself. The version id is the sixth field of the --versions listing.

$ echo "governance-protected version id: $VID"

  governance-protected version id: 1425b7eb-5124-4607-a19f-1ced4c55527c

Task 4 — Restore it, and prove the recovery point is real

A protection test on an object nobody has ever read back is a test of the wrong thing. Restore first.

LAB="$HOME/rbdr-lab-17"
T0=$(date +%s)
mc cp lab/rbdr-governance/backup-0900.tar "$LAB/rbdr-restore-before.tar"
RC=$?
T1=$(date +%s)
echo "restore exit code: $RC" | tee -a "$LAB/result.txt"
echo "restore seconds before deletion: $((T1 - T0))" | tee -a "$LAB/result.txt"

cmp "$LAB/rbdr-b.tar" "$LAB/rbdr-restore-before.tar" \
  && echo "restored object matches the source byte-for-byte" | tee -a "$LAB/result.txt"
sha256sum "$LAB/rbdr-restore-before.tar"

cmp must print nothing and exit 0, and the digest must equal the one in sha256.0900. That is the baseline: at this moment the bucket holds a recovery point that has been restored and verified, not merely uploaded.

Task 5 — The failing case: a plain version delete

LAB="$HOME/rbdr-lab-17"
VID=$(mc ls --versions lab/rbdr-governance/backup-0900.tar | awk '{print $6}' | head -1)
mc rm --version-id "$VID" lab/rbdr-governance/backup-0900.tar
echo "plain version delete exit code: $?" | tee -a "$LAB/result.txt"

--version-id names one specific version, and it is the flag form the capture used against this bucket.

$ mc rm --version-id 1425b7eb-5124-4607-a19f-1ced4c55527c lab/rbdr-governance/backup-0900.tar

  mc: <ERROR> Failed to remove `lab/rbdr-governance/backup-0900.tar`. Object, 'backup-0900.tar (Version ID=1425b7eb-5124-4607-a19f-1ced4c55527c)' is WORM protected and cannot be overwritten
>>> exit code: 1

This is the result that misleads people. Stop the experiment here and governance mode looks exactly as strong as compliance mode: same message, same exit code, same administrator turned away.

Task 6 — The same delete, one flag longer

LAB="$HOME/rbdr-lab-17"
VID=$(mc ls --versions lab/rbdr-governance/backup-0900.tar | awk '{print $6}' | head -1)
mc rm --bypass --version-id "$VID" lab/rbdr-governance/backup-0900.tar
echo "bypass delete exit code: $?" | tee -a "$LAB/result.txt"

mc ls --versions lab/rbdr-governance/backup-0900.tar
echo "post-delete listing exit code: $?" | tee -a "$LAB/result.txt"

$ mc rm --bypass --version-id 1425b7eb-5124-4607-a19f-1ced4c55527c lab/rbdr-governance/backup-0900.tar

  Removed `lab/rbdr-governance/backup-0900.tar` (versionId=1425b7eb-5124-4607-a19f-1ced4c55527c).
>>> exit code: 0

--- did the governance-protected object survive? ---

>>> exit code: 0

Removed ... at exit code 0, and the versioned listing that follows is empty at exit code 0 as well. That empty listing is the important half: this was not a delete marker laid over a surviving version, which is what an ordinary mc rm produces on a versioned bucket. There is nothing underneath. The retain-until date was still ahead — the refusal in Task 5, seconds earlier, is the proof that the retention was live at the moment the bypass delete succeeded.

Task 7 — Ask for the restore again

LAB="$HOME/rbdr-lab-17"
mc cp lab/rbdr-governance/backup-0900.tar "$LAB/rbdr-restore-after.tar"
echo "post-deletion restore exit code: $?" | tee -a "$LAB/result.txt"
ls -l "$LAB/rbdr-restore-after.tar" 2>&1 | tee -a "$LAB/result.txt"

The same command that succeeded in Task 4 now fails, and no local file is written. Between the two runs nothing changed except one flag on one command issued by an identity that was always allowed to hold it.

Task 8 — Put it beside the compliance result

$ mc rm --bypass --version-id 133fd99f-1f98-41c0-9d08-95e6e2944157 lab/rbdr-immutable/backup-0900.tar

--- can the FULL ADMIN force it? ---
mc: <ERROR> Failed to remove `lab/rbdr-immutable/backup-0900.tar`. Object, 'backup-0900.tar (Version ID=133fd99f-1f98-41c0-9d08-95e6e2944157)' is WORM protected and cannot be overwritten
>>> exit code: 1

Validation

The pair that constitutes the result is rows six and seven: exit 1 without the flag, exit 0 with it, from one identity against one version. If row seven returns 1, the alias in use does not hold the bypass right and the lab has measured a permissions gap rather than a retention mode.

Expected Outcome

Troubleshooting

Cleanup

rbdr-governance can be removed because its only version was destroyed in Task 6. rbdr-immutable cannot be, and the diff below is what proves it: the compliance bucket appears in both files, unchanged, because no command available to you will move it.

LAB="$HOME/rbdr-lab-17"
mc rb --force lab/rbdr-governance
echo "bucket removal exit code: $?"
rm -f "$LAB/rbdr-b.tar" "$LAB/rbdr-restore-before.tar" "$LAB/rbdr-restore-after.tar"

{
  mc --version
  echo '--- rbdr- buckets present ---'
  mc ls lab/ | awk '{print $NF}' | sed -n 's#/$##p' | grep '^rbdr-' | sort
} | tee "$LAB/state.post-lab"

diff "$LAB/state.pre-lab" "$LAB/state.post-lab" \
  && echo "CLEAN: post-lab state matches the baseline recorded in Task 1"

state.pre-lab, state.post-lab, sha256.0900 and result.txt are deliverables and are kept deliberately; none matches the rbdr-* bucket glob, so the assertion still holds.

Production notes

• Governance mode protects against mistakes. Compliance mode protects against an adversary holding administrative credentials. Only the second is an answer to the ransomware question, because the intruder who took the administrator account took the bypass right in the same motion. A control an administrator can lift is a control the intruder has already lifted.
• Do not read the exit-1 in Task 5 as evidence of strength. It is a real and useful refusal — it stops the mistaken prefix, the cleanup script, the operator under pressure — and it is produced by a mechanism with a documented door through it. The measurement that separates the two modes is only ever the one taken with the flag.
• The choice is made at mc mb, not later. Because object lock is fixed at bucket creation, “start with governance and tighten afterwards” is not a migration, it is a re-write of every object into a new bucket, paid for twice while the move is in flight.
• What compliance mode costs is the mirror image of what it gives. Every identity the capture tried — including the full administrator holding --bypass — was refused the early removal. Pick the period from the obligation, rehearse it with a short one, and budget the storage for its full length before the first object lands.
• These behaviours were executed against MinIO RELEASE.2025-09-07T16-13-09Z with mc RELEASE.2025-08-13T08-35-41Z. S3-compatible implementations do not all share identical semantics: whether compliance mode is offered at all, whether the retain-until date is enforced on every path including lifecycle expiry, and what the refusal text looks like are per-endpoint facts. Run this eight-task probe against the endpoint you intend to trust and keep the transcript beside the design document. A monitoring rule that greps for is WORM protected will not fire on an endpoint that answers 403 Forbidden.

What You Learned

• The two modes are configured identically and promise different things. GOVERNANCE 3DAYS and COMPLIANCE 3DAYS differ by one word in a diff and by everything in a threat model.
• One flag separated a surviving recovery point from a destroyed one. The same administrator, the same version id and the same command produced exit 1 without --bypass and exit 0 with it, with the retain-until date still ahead in both attempts.
• An empty mc ls --versions is a different result from a delete marker. The bypass delete removed the version itself; there was nothing underneath to recover, which is why Task 7’s restore failed where Task 4’s succeeded.
• Restore before you test the protection. Task 4 turned “an object was uploaded” into “a verified recovery point existed at 09:00”, which is the only reason Task 7’s failure can be described as a loss rather than a guess.
• Provenance is part of the finding. Every exit code here belongs to one MinIO release and one mc release; the probe is cheap, and running it is the difference between a verified backup destination and a hoped-for one.