DaemonSet scheduling — nodeSelector, taints, and tolerations

A DaemonSet Pod is scheduled by the DaemonSet controller, not the kube-scheduler. The controller iterates over all nodes that match the DaemonSet’s nodeSelector and creates one Pod per node. If a node has taints the Pod does not tolerate, the Pod is not scheduled onto that node. This lesson covers the configuration that makes a DaemonSet land where it should — and where it shouldn’t.

The DaemonSet controller’s scheduling model

flowchart LR
    DS[DaemonSet controller] --> N1["node-01<br/>worker"]
    DS --> N2["node-02<br/>worker"]
    DS --> N3["node-03<br/>control-plane"]
    DS --> N4["node-04<br/>cordoned"]
    N1 -->|selector matches<br/>tolerates taints| P1[Pod]
    N2 -->|selector matches<br/>tolerates taints| P2[Pod]
    N3 -->|selector matches<br/>tolerates taints?| P3{?}
    N4 -->|cordoned| X[no Pod]

The controller picks a node, evaluates the Pod spec’s nodeSelector and tolerations, and either creates the Pod or skips. The nodeName field of the Pod is set to the chosen node. The scheduler is bypassed.

nodeSelector

spec:
  template:
    spec:
      nodeSelector:
        node-role.kubernetes.io/worker: ""
        disk: ssd

The Pod lands on nodes with all of these labels. The DaemonSet’s per-node count is the count of nodes with the selector match.

A common pattern: the log collector runs on worker nodes only; the metrics collector runs on every node including control-plane. Two DaemonSets, two selectors.

# Worker-only
spec:
  template:
    spec:
      nodeSelector:
        node-role.kubernetes.io/worker: ""

# Every node
spec:
  template:
    spec:
      # no nodeSelector

Taints and tolerations

Control-plane nodes have the taint node-role.kubernetes.io/control-plane:NoSchedule (or node-role.kubernetes.io/master:NoSchedule on older clusters). A Pod that does not tolerate this taint is not scheduled onto control-plane nodes.

For a DaemonSet that must run on every node (e.g., a CNI agent), the Pod tolerates the control-plane taint:

spec:
  template:
    spec:
      tolerations:
      - key: node-role.kubernetes.io/control-plane
        operator: Exists
        effect: NoSchedule
      - key: node-role.kubernetes.io/master
        operator: Exists
        effect: NoSchedule

Or, more loosely, tolerate all taints:

spec:
  template:
    spec:
      tolerations:
      - operator: Exists

The operator: Exists with empty key matches every taint. This is the safe default for agents that must run on every node.

nodeAffinity

The DaemonSet’s Pod template supports nodeAffinity in addition to nodeSelector. This is needed for more complex matching:

spec:
  template:
    spec:
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              - key: kubernetes.io/arch
                operator: In
                values: ["amd64"]
              - key: topology.kubernetes.io/zone
                operator: In
                values: ["us-east-1a", "us-east-1b"]

The Pod lands only on amd64 nodes in the listed zones. DaemonSets for amd64-only images must filter for amd64; an ARM cluster has nodes with kubernetes.io/arch: arm64, and the wrong image on ARM is a crashloop.

schedulerName

By default, the DaemonSet controller bypasses the scheduler. The schedulerName field on the Pod template can be set to delegate to a custom scheduler:

spec:
  template:
    spec:
      schedulerName: my-custom-scheduler

This is rare. Production DaemonSets use the default controller-based scheduling.

Cordoning and unschedulable nodes

A cordoned node (kubectl cordon or --unschedulable=true in the node spec) is not schedulable for normal Pods. The DaemonSet controller honours unschedulable: true for new Pods but does not remove existing DaemonSet Pods.

flowchart TB
    A[kubectl cordon node-04] --> B["node-04 spec.unschedulable=true"]
    B --> C[kube-scheduler ignores node-04]
    B --> D[DaemonSet controller skips node-04 for new Pods]
    B --> E[Existing DaemonSet Pod on node-04 still runs]

To remove a DaemonSet Pod from a cordoned node, the operator must kubectl drain (which evicts the Pod) or kubectl delete pod directly.

Inspecting node placement

kubectl get pods -l app=log-collector -n logging -o custom-columns=\
  NAME:.metadata.name,NODE:.spec.nodeName,READY:.status.conditions[?(@.type=="Ready")].status

flowchart LR
    A[Operator changes selector] --> B[DaemonSet controller]
    B --> C{For each node}
    C -->|selector matches| D{Node taints<br/>tolerated?}
    D -->|yes| E[Create Pod]
    D -->|no| F[Skip node]
    C -->|selector does not match| F

kubectl get nodes --show-labels | grep node-role
# node-01   Ready    control-plane
# node-02   Ready    control-plane,worker
# node-03   Ready    worker

A nodeSelector of node-role.kubernetes.io/worker: "" matches node-02 and node-03 but not node-01.

Quiz

Production discipline

• Tolerate the control-plane taints. Every DaemonSet that needs to run on control-plane nodes must tolerate node-role.kubernetes.io/control-plane:NoSchedule (and legacy node-role.kubernetes.io/master:NoSchedule).
• Filter by architecture. A DaemonSet image built for amd64 on an ARM node is a crashloop. Filter with kubernetes.io/arch in nodeAffinity.
• Use nodeSelector for clean placement rules. Avoid operator: Exists tolerations on DaemonSets that should not run on every node; the cluster-wide blast radius is real.
• Treat cordoned nodes as a deliberate choice. A DaemonSet Pod stuck on a cordoned node is a leftover; drain or delete it.
• Audit the selector. A nodeSelector that matches the wrong nodes — e.g., accidentally including role: experimental — produces a DaemonSet on every node the operator didn’t intend.

DaemonSet scheduling looks simple; the failure modes are silent. A DaemonSet that runs on 99 of 100 nodes is rarely caught without an explicit “count = node count” alert. Operators who run DaemonSets verify the count after every configuration change.