kubectl delete — propagation, foreground, background, orphans

kubectl delete is the most dangerous command in the operator’s toolkit. It cascades by default, and it can block on finalizers silently. This lesson covers the three propagation modes, how garbage collection works, and the production discipline that keeps deletion from becoming an outage.

The three propagation modes

kubectl delete deployment web                            # default: background
kubectl delete deployment web --cascade=background       # explicit default
kubectl delete deployment web --cascade=foreground       # wait for dependents
kubectl delete deployment web --cascade=orphan           # preserve dependents

The --cascade flag (formerly --cascade=true|false; now background|foreground|orphan) controls what happens to objects that are owned by the deleted object.

Background (default): kubectl sends the DELETE request and returns immediately. The garbage collector walks the ownership tree asynchronously and deletes dependents in the background. kubectl does not wait.

Foreground: kubectl sends the DELETE request but the garbage collector marks the object as “being deleted” and deletes dependents first. The API server holds the request until all dependents are gone, then the parent is deleted.

Orphan: kubectl sends the DELETE request and the dependents are removed from the parent’s ownerReferences but not deleted. The dependents survive; they now have no owner (or a different owner if re-attached).

flowchart TD
    Parent[Deployment web] -->|owns| ReplicaSet[ReplicaSet web-7c8]
    ReplicaSet -->|owns| Pod[Pod web-7c8-abc]
    
    B[Background: delete all async] -.->|after| Parent
    F[Foreground: delete children first] -.->|before| Parent
    O[Orphan: detach children, no delete] -.->|detach| Parent

When to use each mode

Background (default): the standard case. Deleting a Deployment should also delete its ReplicaSets and Pods. The cascading happens asynchronously; kubectl returns immediately.

Foreground: when you need the deletion to be visible and atomic. Use cases:

• Deleting a CRD whose CRs have finalizers that must run.
• Deleting a Helm release whose hooks need to be torn down in order.
• When you want to verify the deletion before the parent is removed.

Foreground is slower but more predictable.

Orphan: when you want to preserve dependents. Use cases:

• Promoting a Pod out from under a ReplicaSet (so the Pod is not deleted when the ReplicaSet is scaled down).
• Migrating a StatefulSet’s PVCs to a new StatefulSet.
• Recovery scenarios: detach children so you can edit the parent safely without losing the children.

Garbage collection under the hood

Kubernetes implements owner-based garbage collection as a controller in the controller-manager. It watches for the removal of objects and walks the ownerReferences tree.

flowchart LR
    API[API server] -->|mark deleted| GC[Garbage collector]
    GC -->|find ownerReferences| Children[Children]
    GC -->|cascade delete| Children
    GC -->|mark parent deleted| Parent[Parent]

The API server marks the parent object with a deletion timestamp (metadata.deletionTimestamp). The garbage collector sees this timestamp and starts deleting dependents. For foreground propagation, the controller holds the parent’s finalizers until the children are gone.

When all dependents are gone and all finalizers are removed, the parent object is actually removed from etcd. Until then, the object exists in a “being deleted” state — it is read-only and excluded from most queries.

Finalizers and stuck deletes

A finalizer is a string key in metadata.finalizers. When present, the API server will not actually delete the object until the finalizer is removed. Finalizers are how controllers get a chance to do cleanup work (release external resources, notify external systems) before the object disappears.

kubectl get pod web-7c8 -o yaml | grep -A 5 finalizers

A Pod stuck in Terminating:

kubectl get pod web-7c8
# NAME      STATUS        AGE
# web-7c8   Terminating   5m

kubectl get pod web-7c8 -o yaml | grep -A 3 finalizers
# finalizers:
# - kubernetes.io/pv-protection

The Pod is being deleted but the pv-protection finalizer is blocking it because the Pod’s volumes have not been released. The fix is to release the volumes first, then the finalizer will be cleared automatically.

Common finalizers:

• kubernetes.io/pv-protection — blocks PVC deletion while Pods reference it.
• kubernetes.io/pvc-protection — blocks PV deletion while PVCs reference it.
• foregroundDeletion — set by the foreground propagation cascade.
• Custom finalizers added by controllers, CRDs, service meshes, etc.

Forcing a deletion past finalizers:

kubectl delete pod web-7c8 --grace-period=0 --force
kubectl delete pod web-7c8 --grace-period=0 --force --cascade=orphan

--force removes the finalizers from the API server before the deletion completes. This bypasses the cleanup logic that the finalizer was supposed to run. This is dangerous: the external resource (a PVC, a cloud LB, a CSI volume) is left orphaned. Use it only when:

• The finalizer’s controller is broken (no chance of recovery).
• The associated external resource has been manually cleaned up.
• You have documented the cleanup and understand the consequences.

Deletion timestamps and graceful termination

kubectl delete sends a graceful termination request by default. The kubelet receives the request, sets the Pod’s deletion timestamp, runs any preStop hooks, and sends SIGTERM to the containers. After the terminationGracePeriodSeconds (default 30s), the kubelet sends SIGKILL and removes the Pod.

The grace period is configurable:

kubectl delete pod web-7c8 --grace-period=60
kubectl delete pod web-7c8 --grace-period=0 --force    # immediate SIGKILL

For Pods with sidecars, the grace period must account for the sidecar’s shutdown time. Production discipline: set terminationGracePeriodSeconds on the Pod to account for the slowest container’s shutdown.

Deletion blast radius

A single delete can cascade to many objects. The production discipline:

1. Confirm what will be deleted before the command.

   # Substitute your own values before running:
   KIND=deployment
   NAME=checkout-api
   
   kubectl get "$KIND"/"$NAME" --output=jsonpath='{.metadata.ownerReferences}' | jq

   This shows who owns the object. To see what the object owns:

   # Substitute your own values before running:
   KIND=deployment
   NAME=checkout-api
   CHILD_KIND=replicaset
   
   kubectl get "$KIND"/"$NAME" -o json | jq '.metadata.uid, [.metadata.name]'
   # Then for each kind, matching on the parent uid printed above:
   PARENT_UID=$(kubectl get "$KIND"/"$NAME" -o jsonpath='{.metadata.uid}')
   kubectl get "$CHILD_KIND" -A -o json | jq --arg uid "$PARENT_UID" '.items[] | select(.metadata.ownerReferences[].uid==$uid) | .metadata.name'

2. Delete in dependency order when possible. Delete Pods, then ReplicaSets, then Deployments. This is slower than cascade but predictable.
3. Use --dry-run=server to confirm the request is accepted before sending it.
4. Capture the object before deletion. A deletion is irreversible except via etcd restore.

Cross-course references

• The Linux course part XXII-Linux-NetTroubleshoot covers cleanup discipline at the OS level; the same logic applies to cluster objects.
• The Ansible course part XXXVI-Ansible-Drift covers cleanup of drifted state; cluster garbage collection is the Kubernetes equivalent.
• The Terraform course part XXVIII-Terraform-Disaster-Recovery covers recovery from destructive operations; etcd snapshot restore is the equivalent for cluster state.

Quiz

Production discipline

• Use --cascade=orphan when you want to preserve dependents. Promotions, migrations, recovery scenarios.
• Foreground propagation for predictable deletion. When you need the deletion to be atomic and ordered.
• Investigate finalizers before forcing past them. A Pod stuck in Terminating is signalling a real problem; the fix is to clear the resource the finalizer is protecting, not to bypass it.
• Capture the object before deletion. A deletion is irreversible except via etcd restore. kubectl get -o yaml is the cheap insurance.
• Make deletion a runbook step. Ad-hoc deletions in production are an outage risk. The runbook entry should cover cascade, blast radius, rollback, and verification.