Skip to main content
RunBook Academy

KubernetesCXIX · Pod TroubleshootingPod troubleshooting

Stuck Terminating — the eviction and shutdown diagnostic

Advanced⏱ ~15 minkubectl

What you'll learn

  • Apply the 11-step methodology to a Pod stuck in Terminating
  • Identify the finalizer, the grace period, and the preStop hook as the cause
  • Force-delete the Pod with --force --grace-period=0
  • Identify the production failure modes of stuck Terminating

Prerequisites

Verified against Kubernetes 1.34.x · kubeadm 1.34.x · kubectl 1.34.x · etcd 3.6.x · CoreDNS 1.11.x · containerd 1.7.x / 2.x · 2026-08-16

Not yet marked complete on this device.

A Pod stuck in Terminating is the kubelet waiting for the container to shut down. The grace period, the preStop hook, and the finalizer are the cause. The force-delete is the remediation. The discipline is to identify the cause before applying the force-delete.

The Terminating lifecycle

A Pod that is being deleted goes through the Terminating state. The kubelet:

  1. Sends SIGTERM to the container.
  2. Waits for the grace period (default 30s).
  3. Sends SIGKILL if the container is still running.
  4. Removes the Pod from the API.
flowchart TD
    A[Pod deleted] --> B[Send SIGTERM]
    B --> C{Container exits?}
    C -->|Yes| D[Remove from API]
    C -->|No within grace period| E[Send SIGKILL]
    E --> D
    D --> F[Pod gone]

A Pod that is stuck in Terminating is one where the kubelet cannot complete the lifecycle. The cause is one of:

  • The container is not responding to SIGTERM.
  • The preStop hook is hanging.
  • A finalizer is preventing the API from removing the Pod.

The diagnostic

A real kubectl describe pod for a stuck Terminating Pod:

Name:         billing-7d8f-abcde
Namespace:    prod
Status:       Terminating
Termination Grace Period:  30s
Controlled By:  Deployment/billing
Containers:
  billing:
    Image:    registry.example.com/billing:1.2.3
    State:    Running
      Started:  Fri, 16 Aug 2026 04:23:01 +0000
    Ready:    True
Conditions:
  Type           Status
  Initialized    True
  Ready          True
  ContainersReady  True
  PodScheduled   True
Events:
  Type     Reason          Age   From               Message
  ----     ------          ----  ----               -------
  Normal   Killing         4m    kubelet            Stopping container billing

The events tell the operator:

  • The kubelet sent SIGTERM 4 minutes ago.
  • The container is still running.
  • The grace period is 30s, but the Pod is still Terminating.

The diagnostic is the API object’s finalizers:

kubectl get pod billing-7d8f-abcde -n prod -o yaml | grep -A5 finalizers

The output may show:

metadata:
  finalizers:
  - billing.example.com/cleanup

The finalizer is preventing the API from removing the Pod.

The remediation

The remediation depends on the cause:

  1. Container not responding to SIGTERM. The application has a bug that ignores SIGTERM. Force-delete the Pod and fix the application.
  2. preStop hook hanging. The preStop hook is hanging (e.g., waiting for a network connection). Force-delete the Pod and fix the preStop hook.
  3. Finalizer blocking. The finalizer is preventing the API from removing the Pod. Force-delete the Pod and remove the finalizer.
kubectl delete pod billing-7d8f-abcde -n prod --force --grace-period=0

The --force flag bypasses the kubelet’s lifecycle. The --grace-period=0 shortens the grace period to 0. The Pod is removed from the API immediately.

Why Pods get stuck

The most common causes:

  • Application bug. The application ignores SIGTERM. The force-delete is the only remediation.
  • Long preStop hook. The preStop hook waits for a network connection that does not arrive. The force-delete is the only remediation.
  • Finalizer. A controller adds a finalizer to the Pod (e.g., for cleanup). The controller is failing to remove the finalizer. The force-delete bypasses the controller.
  • Kubelet is unhealthy. The kubelet cannot send the SIGTERM. The Pod is stuck at the kubelet level. The fix is the kubelet.

Production discipline

A Pod stuck in Terminating is a workload that will not shut down. The discipline is to identify the cause, fix the underlying issue, and only then force-delete. The force-delete is a fix, not a procedure.

  • Identify the cause first. The events and the finalizers tell the operator.
  • Investigate the controller. A finalizer is owned by a controller; the controller’s health is the diagnostic.

Quiz

Knowledge check · 4 questions

  1. Q1. What is the difference between a finalizer and a preStop hook?

  2. Q2. Force-deleting a Pod is a routine operation that should be the first response to a stuck Terminating Pod.

  3. Q3. An operator runs `kubectl get pod billing-7d8f-abcde -n prod`. The Pod is in `Terminating` state for 8 minutes. The kubelet logs show `Stopping container billing`. The Pod's finalizers include `billing.example.com/cleanup`. What is the diagnostic and remediation?

    The Pod is `billing-7d8f-abcde` in namespace `prod`. The workload is a 6-replica Deployment. The Pod has been Terminating for 8 minutes. The kubelet is the same version as the cluster. The Dep

  4. Q4. Name three common causes of a Pod stuck in Terminating and the diagnostic command for each.

Passing score: 75%. Answers are checked in this browser.