preStop hooks — what to do before SIGTERM arrives

The preStop hook is a chance to do work before the kubelet sends SIGTERM. This lesson covers the three hook types, the production patterns where a preStop hook helps, and the anti-patterns that are common in the field.

Hook types

lifecycle:
  preStop:
    exec:
      command: ["sh", "-c", "nginx -s quit"]

Three hook types:

• exec: run a command inside the container’s filesystem. The hook runs as the container’s user; the command has access to the container’s environment.
• httpGet: send an HTTP GET to a path/port on the container. The kubelet makes the request directly (not through the CNI).
• tcpSocket: open a TCP connection to a port. Success is connection established.

The hook must complete within the kubelet’s hook timeout (default — there is no separate hook timeout; the hook counts against terminationGracePeriodSeconds).

When the hook runs

sequenceDiagram
    participant API as API server
    participant Kubelet
    participant App as Application<br/>(PID 1)
    participant Hook as preStop hook

    API->>Kubelet: watch sees deletionTimestamp
    Kubelet->>Hook: run preStop hook (synchronous)
    Hook-->>Kubelet: hook completes
    Kubelet->>App: SIGTERM
    Note over App: graceful shutdown begins
    Kubelet->>App: SIGKILL (if grace expires)

The preStop hook runs synchronously before SIGTERM. The kubelet waits for the hook to complete (or for the grace period to expire) before sending SIGTERM.

Use cases for preStop

Service deregistration via HTTP:

lifecycle:
  preStop:
    httpGet:
      path: /admin/drain
      port: 8080

The application implements /admin/drain which sets an internal flag marking the Pod as draining. The kubelet calls this endpoint; the application stops accepting new requests immediately. SIGTERM follows; the application exits cleanly.

This pattern is faster than waiting for the readiness probe to fail because the kubelet calls the hook directly, not via the Endpoints controller.

State flush to disk:

lifecycle:
  preStop:
    exec:
      command: ["sh", "-c", "sync && echo 3 > /proc/sys/vm/drop_caches"]

Flush filesystem buffers before SIGTERM. The application doesn’t need to know about SIGTERM; the preStop hook guarantees a clean filesystem state at exit.

Signal forwarding to a non-PID-1 process:

lifecycle:
  preStop:
    exec:
      command: ["sh", "-c", "kill -TERM $(pgrep -f my-app)"]

If the application’s main process is not PID 1 (e.g., an init script starts the actual app), the preStop hook can forward SIGTERM to the right process. The kubelet’s SIGTERM goes to PID 1; the hook reaches the actual app.

Drain delay via the Endpoints controller

The Endpoints update lag has three components:

• Probe period (periodSeconds on the readiness probe): the kubelet runs the probe this often.
• Endpoints controller reconciliation: typically a few seconds.
• kube-proxy rule propagation: typically a few hundred milliseconds.

For a readiness probe with periodSeconds: 10 and failureThreshold: 3, the worst-case lag is 30s. The Endpoints may continue to list the Pod for up to 30s after the readiness probe starts failing.

To drain connections cleanly, the application’s shutdown sequence should:

1. Catch SIGTERM.
2. Set ready=False (via the readiness probe).
3. Sleep for probe period * failure threshold + controller lag + kube-proxy lag (typically 35-40s).
4. Exit.

This is the canonical graceful shutdown. The preStop hook is not used for the drain delay — the application handles it.

When preStop is the wrong tool

• Drain delay: implement in the application (via SIGTERM handler + readiness probe).
• Long shutdown: extend terminationGracePeriodSeconds instead.
• Configuration changes: not a termination concern.
• Coordination with other Pods: preStop is per-Pod; for cluster-wide coordination use PodDisruptionBudgets.

preStop and sidecars

With native sidecars (Part XI-03), the preStop hook order is:

1. Main container’s preStop runs (if defined).
2. Main container’s SIGTERM is sent.
3. Sidecar’s preStop runs.
4. Sidecar’s SIGTERM is sent.

The order matters: the main container can drain traffic while the sidecar is still running, then both terminate in order. Pre-1.28 (with init container-based sidecars), the sidecar terminated after the main container but in non-deterministic order.

Production patterns

HTTP drain endpoint:

lifecycle:
  preStop:
    httpGet:
      path: /admin/drain
      port: 8080

The application implements /admin/drain to mark itself as draining. The kubelet calls the hook; the application stops accepting new requests immediately. The readiness probe should also fail during draining so the Endpoints controller removes the Pod from Endpoints.

Multi-step drain with sleep:

lifecycle:
  preStop:
    exec:
      command:
      - sh
      - -c
      - |
        # Deregister from service registry
        curl -X POST http://localhost:8080/admin/drain
        # Wait for Endpoints controller to update
        sleep 10
        # Flush state
        sync

The hook calls the drain endpoint, sleeps 10s, and flushes the filesystem. The application sees SIGTERM after the hook completes (or after the grace period expires).

Signal forwarding:

lifecycle:
  preStop:
    exec:
      command: ["sh", "-c", "kill -TERM $(pgrep -f my-app)"]

If the main process is not PID 1, the hook forwards SIGTERM to the actual application. This pattern is common in legacy images where the entrypoint is a wrapper script.

Failure modes

• Hook times out: if the hook exceeds terminationGracePeriodSeconds, the kubelet proceeds to SIGTERM. The hook’s work is not interrupted, but the application gets SIGTERM with less time to drain.
• Hook fails (exit code non-zero for exec, non-2xx for httpGet): the kubelet logs a warning but proceeds to SIGTERM. The hook’s failure does not block termination.
• Hook hangs (e.g., network call to a slow endpoint): the kubelet waits for the hook to complete or for the grace period to expire. If the grace period expires first, SIGTERM is sent; the hook continues running in parallel.

Production discipline: hooks must be fast and reliable. A preStop that depends on an external service is a risk; if the service is slow, the hook holds up termination.

Cross-course references

• The Linux course part VI-Linux-Processes covers process shutdown patterns; preStop is the cluster-level equivalent of an init script’s pre-shutdown step.
• The Docker course part XXX-Docker-Lifecycle covers container shutdown; preStop is the cluster-level extension.
• The Ansible course part XXXV-Ansible-Scripting covers service deregistration; the HTTP drain pattern is the same idea.

Quiz

Production discipline

• Use preStop for Service deregistration and state flush. The drain endpoint pattern is the standard.
• Avoid sleep N anti-patterns. Implement graceful shutdown in the application instead.
• Make hooks fast and reliable. A preStop that hangs delays SIGTERM and wastes the grace period.
• Signal-forward via exec hook when the application is not PID 1. The kubelet sends SIGTERM to PID 1 only.
• Test preStop under load. Trigger Pod deletions, verify the timeline of drain + readiness update + SIGTERM + in-flight completion.