Startup probes — slow-starting containers and the InitialDelay trap

A slow-starting container needs a different kind of probe than a steady-state container. Startup probes are designed for exactly this case: while the startup probe is failing, the kubelet does not run the readiness or liveness probes. Once the startup probe succeeds, the kubelet starts running the readiness and liveness probes on the regular schedule.

The probe lifecycle

A Pod has three independent probes per container:

• startupProbe: runs first, on its own schedule.
• readinessProbe: starts running once startupProbe succeeds; determines if the Pod receives traffic.
• livenessProbe: starts running once startupProbe succeeds; determines if the container is restarted.

stateDiagram-v2
    [*] --> Startup: container created
    Startup --> Startup: startup probe failing
    Startup --> Running: startup probe succeeds
    Running --> Ready: readiness probe succeeds
    Ready --> NotReady: readiness probe fails
    NotReady --> Ready: readiness probe succeeds
    Ready --> Restarted: liveness probe fails (restart)
    Restarted --> Startup: container restarts

The startup probe is the gate. Until it succeeds, the readiness and liveness probes are disabled. This is the crucial property: the kubelet does not restart the container during startup just because the liveness probe is configured but not yet passing.

When startup probes are needed

The use cases:

• JVM applications with warm-up time (JIT, class loading): 30s-2min before serving traffic.
• Applications that load large datasets at startup (in-memory caches, ML model loading): 10s-60s.
• Migrations or schema updates at startup: depends on the migration duration.
• Network daemons that bind on a port during startup but aren’t ready until DNS is reachable.
• Anything where readiness takes longer than the liveness probe’s period: the typical case.

The signature pattern:

• The application becomes ready after T seconds.
• The liveness probe has periodSeconds: 10, failureThreshold: 3 (so it kills the container after 30s of failure).
• Without a startup probe, the liveness probe kills the container during startup.

The fix: add a startup probe with failureThreshold * period greater than T.

Configuring a startup probe

startupProbe:
  httpGet:
    path: /healthz
    port: 8080
  failureThreshold: 30
  periodSeconds: 5

This probe allows 30 failures × 5s = 150s for startup. If the application becomes ready within 150s, the probe eventually succeeds; then the readiness and liveness probes take over.

The probe types:

• httpGet: GET request to a path/port; success = 2xx or 3xx response.
• tcpSocket: open a TCP connection; success = connection established.
• exec: run a command; success = exit code 0.
• grpc: gRPC health check (since 1.24).

The probe fields:

• initialDelaySeconds: delay before the first probe. Default 0. For startup probes, often 0 is fine because the probe’s failureThreshold * periodSeconds gives the total window.
• periodSeconds: interval between probes. Default 10.
• timeoutSeconds: per-probe timeout. Default 1.
• failureThreshold: consecutive failures before the probe is considered failed. Default 3.
• successThreshold: consecutive successes for the probe to be considered successful. Default 1 (must be 1 for liveness and startup probes).

flowchart LR
    Probe[Probe runs] -->|succeeds| S[Count = 0]
    Probe -->|fails| F[Failure count++]
    F -->|"count > failureThreshold"| State[Container marked failed]
    S -->|next probe| Probe

The total window for a startup probe: failureThreshold * periodSeconds. For 30 × 5s = 150s, the application has 150s to become ready.

The InitialDelay trap

The naive pattern (without startupProbe):

livenessProbe:
  httpGet:
    path: /healthz
    port: 8080
  initialDelaySeconds: 120  # wait 2 minutes before first probe
  periodSeconds: 10
  failureThreshold: 3

This pattern has two problems:

1. It hides the real readiness time. If the application sometimes starts in 30s and sometimes in 90s, the 120s initialDelaySeconds wastes 30-90s of probe coverage for the slow case.
2. It doesn’t help if startup time exceeds the delay. If the application sometimes takes 150s, the initialDelay is not enough and the liveness probe kills the container.

The startup probe replaces initialDelaySeconds:

startupProbe:
  httpGet:
    path: /healthz
    port: 8080
  failureThreshold: 30
  periodSeconds: 5
livenessProbe:
  httpGet:
    path: /healthz
    port: 8080
  periodSeconds: 10
  failureThreshold: 3

The startup probe gives up to 150s for startup, regardless of how long the application actually takes. The liveness probe takes over after success, with normal sensitivity.

Startup probes and readiness

Once the startup probe succeeds, the readiness probe takes over. The Pod transitions to Ready when the readiness probe succeeds. If the readiness probe has its own initialDelaySeconds, that delay applies after the startup probe succeeds, not from container start.

startupProbe:
  httpGet:
    path: /healthz
    port: 8080
  failureThreshold: 12
  periodSeconds: 5
readinessProbe:
  httpGet:
    path: /ready
    port: 8080
  initialDelaySeconds: 0   # relative to startup probe success
  periodSeconds: 5
  failureThreshold: 3

Total time from container start to Ready: up to 12 * 5 + 0 + 3 * 5 = 75s worst case. Adjust as needed.

Startup probes and native sidecars

Native sidecars (Part XI-03, introduced in Kubernetes 1.28) have their own startup sequence. The kubelet waits for the sidecar’s startupProbe to succeed before starting the main container. This is the standard pattern for sidecars that must be running before the main container can do its work (e.g., a service mesh proxy).

initContainers:
- name: istio-proxy
  image: istio/proxyv2:1.20
  restartPolicy: Always  # native sidecar
  startupProbe:
    httpGet:
      path: /healthz/ready
      port: 15021
    failureThreshold: 30
    periodSeconds: 2

This startup probe gates the main container’s start. Without it, the main container might start before the sidecar is ready, causing connection failures.

Cross-course references

• The Linux course part VI-Linux-Processes covers process startup; startup probes are the cluster-level equivalent of waiting for a daemon to bind on its port.
• The Docker course part XXX-Docker-Lifecycle covers container health checks; startup probes are the cluster-level extension.
• The Ansible course part XXXV-Ansible-Scripting covers service readiness; startup probes are the cluster-level equivalent.

Quiz

Production discipline

• Use startupProbe for slow-starting containers. JVMs, ML model loading, migrations — anything with variable startup time.
• Never crank initialDelaySeconds past 60s on the liveness probe. Use a startupProbe with a high failureThreshold.
• Set startupProbe’s window to actual startup time plus headroom. 1.5-2× the typical startup time.
• Verify probe paths are cheap. The probe runs every periodSeconds; a heavy probe adds load.
• Combine startupProbe with readinessProbe for staged readiness. startupProbe gates the readiness check; the readiness probe then determines when traffic flows.