Binding — how the scheduler reserves a node and commits the choice

The scheduler’s bind step is the commit: it writes the Pod’s spec.nodeName to the API server. Before the bind, the scheduler reserves the node in its own cache so concurrent schedulers do not double-book. This lesson covers the reserve and bind steps, the failure modes, and the recovery path.

Note: typo in the UnderTheHood import below; corrected by the validator.

The reserve step

After the score phase selects a node, the scheduler’s Reserve plugin reserves the node in its in-memory cache:

flowchart LR
    A["Score phase selects<br/>node-01"] --> B[Reserve plugin]
    B --> C["Cache:<br/>Pod X reserved for node-01"]
    C --> D{Concurrent<br/>scheduler?}
    D -->|yes| E["Other scheduler sees<br/>node-01 reserved for X"]
    D -->|no| F[Bind proceeds]

The reserve is internal to the scheduler. Other scheduler instances see the reservation through the scheduler’s cache (assumed Pods). The bind is the external commit.

Assumed Pods

A Pod that the scheduler has reserved but not yet bound is an “assumed Pod.” The scheduler treats it as if it were running on the reserved node:

flowchart TB
    A["Scheduler caches:<br/>Pod X assumed on node-01"] --> B[Filter phase]
    B --> C["Other Pod Y: filter<br/>PodFitsResources"]
    C --> D{node-01 has<br/>Pod X assumed?}
    D -->|yes| E["Subtract X's resources<br/>from node-01's available"]
    D -->|no| F[Full allocatable]

The scheduler subtracts assumed Pods’ resource requests from the node’s allocatable. This prevents over-subscription in a multi-scheduler cluster.

The assumption is optimistic. If the bind fails, the assumption is rolled back; the assumed Pod is no longer counted.

The permit step (intermediate)

For some plugins (VolumeScheduling, others), the scheduler needs to acquire a “permit” before binding. The permit is an asynchronous resource acquisition:

sequenceDiagram
    participant S as Scheduler
    participant V as Volume plugin
    participant A as API server
    S->>V: Permit (reserve volume)
    V->>A: Wait for PV bound
    A-->>V: PV bound
    V-->>S: Permit granted
    S->>S: Bind
    S->>A: Update Pod nodeName

VolumeScheduling is the most common permit-using plugin. The scheduler must wait for the PV to be bound before committing the Pod to the node. The permit step serialises against external resource acquisition.

The bind step

sequenceDiagram
    participant S as Scheduler
    participant A as API server
    participant K as Kubelet
    S->>A: POST /api/v1/namespaces/<ns>/pods/<name>/binding
    Note over S,A: body: { target: { apiVersion: v1, kind: Node, name: node-01 } }
    A->>A: Update Pod spec.nodeName = node-01
    A-->>S: 200 OK
    K->>A: Watch for Pod with nodeName=self
    A-->>K: Pod spec
    K->>K: Start Pod

The scheduler creates a Binding object via the API server. The Binding object is a special API resource (/api/v1/namespaces/<ns>/pods/<name>/binding) that sets the Pod’s nodeName. The API server validates the binding (target Node exists, RBAC allows the scheduler to bind) and updates the Pod.

After the bind succeeds, the kubelet on the chosen node observes the Pod (it watches for Pods with spec.nodeName == self) and starts the container.

Bind failures

A bind can fail for several reasons:

flowchart TB
    A[Bind fails] --> B{Target Node<br/>deleted?}
    A --> C{RBAC denied?}
    A --> E{API server error?}
    C --> F[Scheduler retries]
    E --> F
    B --> G["Pod remains<br/>Pending"]

In all cases, the scheduler retries. The reserve is released; the assumed Pod is rolled back. The Pod may be re-scheduled on a different node.

Pre-bind hooks

Some plugins run a Pre-bind step before the bind is committed. VolumeBinding, for example, may delay the bind until the PV is bound. The plugin holds the assumed Pod until the bind is safe to commit.

flowchart TB
    A[Score selects node] --> B[Reserve]
    B --> C["Pre-bind:<br/>ensure volume bound"]
    C --> D{Bind safe?}
    D -->|yes| E[Bind]
    D -->|no| F[Wait for resource]
    F --> C

The Pre-bind step is the source of “the scheduler is slow” perception. VolumeBinding can wait minutes for a slow provisioner; the Pod is “Pending” the whole time, but the scheduler is actively working.

Bind vs ready

A bound Pod is not a running Pod. The bind step sets the nodeName; the kubelet still needs to:

1. Pull the image.
2. Mount the volumes.
3. Start the container.
4. Pass the readiness probe.

A Pod that is bound but not ready is in Running phase with Ready: False. The operator sees kubectl get pods report the Pod is Running but not Ready.

flowchart LR
    A["Pod Pending<br/>no nodeName"] --> B["Scheduler binds<br/>Pod Pending<br/>nodeName set"]
    B --> C["Kubelet pulls image<br/>Pod Running<br/>Ready False"]
    C --> D["Readiness passes<br/>Pod Running<br/>Ready True"]

Quiz

Production discipline

• Pending Pods without nodeName are scheduler failures. The Pod has not been bound; investigate the scheduler’s events.
• Pending Pods with nodeName are kubelet failures. The Pod has been bound; the kubelet cannot start it (image pull, volume mount, readiness probe).
• The scheduler retries binds automatically. A single bind failure is not a permanent error; the scheduler re-enters the pipeline with a new node.
• VolumeBinding’s Pre-bind can be slow. A slow provisioner is the most common source of “the scheduler is hanging.” The Pod is Pending; the scheduler is waiting for the volume.
• Audit the scheduler’s bind latency. A dashboard that surfaces time_spent_in_scheduler (or the equivalent metric) catches slow binds.

The bind step is the commit that makes the scheduling decision visible. Operators who understand the bind step distinguish scheduler failures from kubelet failures without confusion.