Orphaned resources and the cluster — what survives when the manifest disappears

An orphan is a cluster resource that no GitOps controller tracks. The resource exists in the cluster, the GitOps controller has no metadata that says “this is mine”, and the controller will never prune it, never update it, and never reconcile it. The orphan survives every sync, accumulates over the lifetime of the cluster, and breaks the GitOps promise that Git is the source of truth - because the cluster now contains a resource whose truth lives nowhere.

Why orphans accumulate

Orphans come from four sources. Each one is a small decision that adds up to a long-running cluster where the cluster’s actual state is larger than the union of every Application’s manifests.

flowchart TD
    A["kubectl apply by operator"] --> O["Orphan cluster resource"]
    B["Helm install outside GitOps"] --> O
    C["Disaster recovery runbook"] --> O
    D["Controller decommissioned, resource left behind"] --> O
    O --> E["Cluster > sum of manifests"]
    E --> F["Git is no longer source of truth"]

The four sources share a pattern: the resource was created by a process that did not register the resource with a GitOps controller. Argo CD’s tracking metadata is app.kubernetes.io/instance plus the Application’s UID; Flux’s tracking metadata is the kustomize.toolkit.fluxcd.io/name label. A resource missing both sets of metadata is invisible to both controllers.

What orphans survive

Orphans survive:

• Every prune — the controller does not know they exist.
• Every sync — the controller’s render does not contain them, so the diff produces no deletion action.
• Every drift correction — the controller’s desired state does not mention them, so the self-heal does not touch them.
• Every rename — a rename of the related Application does not affect orphans because they have no Application binding.

The orphan is, in effect, a permanent piece of cluster state that no automation will touch. An engineer with kubectl delete can remove it. An admission webhook can mutate it. A node failure can restart it. But no controller will reconcile it against any declared desired state, and no Git commit records its existence.

Detecting orphans

Orphan detection is a two-command process:

argocd app manifests "$APP_NAME"
kubectl get all -n "$NAMESPACE" \
  -l app.kubernetes.io/instance="$APP_NAME" \
  -o json

A resource in the kubectl output but not in the manifests output is either an orphan or a resource owned by a different controller. The classification step is to check whether the resource carries the controller’s tracking metadata:

kubectl get deploy payment-worker-debug -n payments \
  -o jsonpath='{.metadata.labels}'

A label set that does not include app.kubernetes.io/instance=$APP_NAME and does not include kustomize.toolkit.fluxcd.io/name=$APP_NAME is an orphan from both controllers’ points of view.

The production rules

1. Orphans are documented or deleted within a release. A new Application’s audit must include a sweep for orphans in its target namespace; the team decides whether to claim each one into the chart, delete it, or document its unmanaged status.
2. Off-GitOps operations are forbidden in production. A kubectl apply from an operator’s terminal must be followed by a commit that adds the manifest to the chart. The exception is an incident; the rule is “every resource has a Git home”.
3. Helm installs from CI or runbooks must register the resource with the controller. A chart that installs via Helm outside the GitOps flow is an orphan factory.
4. Orphan detection is a periodic audit, not a one-time cleanup. A quarterly review of every production namespace against every Application’s manifests is the discipline that keeps the Git source-of-truth promise intact.

Cross-course references

• Git, CI/CD & GitOps for Infrastructure Engineers - Part LXXX-02 (Prune versus orphan) is the foundation this lesson extends; Part LXXV (Drift) is the broader divergence family.
• Kubernetes for Production Sysadmins - Part VII (Workloads) is the resource class orphans are; Part IX (Namespaces and Quotas) is where the per-namespace audit operates.

Quiz