Prune safety mechanisms — dry-run, sandbox clusters, and the escape hatch

Three mechanisms turn prune from a destructive operation into a controlled one. The first is the dry-run diff: a preview of what the next sync will delete before the sync runs. The second is the sandbox cluster: a non-production environment where the first prune can be verified end-to-end before it touches production. The third is the Prune=false sync option: an opt-out that prevents the controller from deleting a protected Application or resource, even when pruning is otherwise enabled. The three together are the production-grade prune procedure.

Dry-run diff before every prune-enabled sync

The argocd app diff command renders the new state and prints the diff without applying anything - a preview with no side effects. The output shows every create, update, and delete the next sync would perform.

argocd app diff "$APP_NAME" --server-side-generate

The --server-side-generate flag uses the server-side apply diff path, matching the apply mode the controller will use. The output includes a line for each delete candidate with the resource’s Kind, name, and namespace. A team that diffs before sync and confirms the deletes against an expected list has the audit trail that prevents an unintended delete.

flowchart LR
    A["Run argocd app diff"] --> B["Read the delete lines"]
    B --> C{"Deletions match intent?"}
    C -->|yes| D["Proceed with sync"]
    C -->|no| E["Block and investigate"]
    D --> F["argocd app sync with prune"]
    E --> G["Revert or fix the chart"]

The diff is the cheapest safety mechanism. It runs in seconds, has no side effects, and produces the exact delete list the sync will execute. A team that skips the diff on a prune-enabled sync is choosing to trust the chart without verifying it.

Sandbox clusters for first-prune verification

A sandbox cluster is a non-production Kubernetes cluster where the first prune-enabled sync is run end-to-end. The cluster mirrors production’s structure but contains no real traffic. A successful sandbox sync proves the prune behaves as expected; a failed sync catches the unintended delete before it reaches production.

argocd app sync "$APP_NAME" \
  --prune \
  --server "$SANDBOX_ARGOCD"

The --server flag points at the sandbox Argo CD. The Application name is the same; the cluster, the sync target, and the watching team are different. A sandbox sync that deletes an unintended resource is the cheapest lesson a team can learn about prune, because the cost is in the sandbox and the recovery is a fresh sync from a corrected chart.

The Prune=false escape hatch

The escape hatch for the Application - or the single resource - that must not be deleted, even when pruning is otherwise enabled:

argocd app set "$APP_NAME" \
  --sync-option Prune=false

The flag sets the sync option on the Application’s standing configuration. Syncs respect the override and do not delete; prune candidates are reported as out-of-sync instead. Revert it with argocd app set "$APP_NAME" --sync-option '!Prune=false' (the ! prefix removes the option). For a single resource, the same opt-out is an annotation:

metadata:
  annotations:
    argocd.argoproj.io/sync-options: Prune=false

The Application-level option protects the whole inventory; the annotation protects one resource. Both are reversible - remove the annotation in Git, or run argocd app set with the '!Prune=false' removal prefix. And a manual argocd app sync deletes nothing unless it is passed --prune: per-sync pruning is opt-in, not opt-out.

The combined procedure

The three mechanisms combine into a single procedure:

1. Diff. argocd app diff "$APP_NAME" --server-side-generate. Read the delete lines. Confirm each deletion against the PR.
2. Sandbox sync for the first prune of an Application, or after any chart restructure. Verify the post-sync state.
3. Production sync with the right opt-outs in place. Set --sync-option Prune=false on the Application - or the argocd.argoproj.io/sync-options: Prune=false annotation on the resource - for anything that must never be deleted.

The procedure is not optional. Prune is the largest blast radius in the controller; the procedure is what makes it survivable.

Production discipline

1. Dry-run diff is paired with every prune-enabled sync. The diff is the audit trail for the deletions.
2. The first prune-enabled sync of an Application is run in a sandbox first. Production is not the place to discover that a chart refactor had an unintended consequence.
3. Prune=false is a scoped, reversible opt-out, not an excuse to abandon pruning. A team that disables prune wholesale to protect a single risky resource has lost the prune discipline; annotate the resource and keep the Application pruning.
4. Branch protection is the upstream safety. A typo in the chart must not auto-prune before review.

Cross-course references

• Git, CI/CD & GitOps for Infrastructure Engineers - Part LXXX-01 (What prune is) is the foundation; Part LXXIX-04 (Prune and the blast radius) is the prior lesson that named the discipline.
• Kubernetes for Production Sysadmins - Part XVI (Operators and Controllers) covers the reconcile loop prune extends; Part XXXV (Staging and Pre-prod) is where the sandbox cluster lives.

Quiz