CPU affinity and NUMA - controlling where processes run

CPU affinity binds a process to specific CPUs. NUMA (Non-Uniform Memory Access) adds memory locality on multi-socket hosts. Both can have significant performance impact.

CPU affinity

The kernel scheduler normally moves processes between CPUs to balance load. Affinity fixes a process to specific CPUs.

Benefits:

• Cache locality (CPU caches stay warm).
• Predictable performance.
• NUMA-aware.

Drawbacks:

• Reduced load balancing.
• Underutilised CPUs if affinity is too narrow.

taskset

# Substitute your own values before running:
PID=1234

# View current affinity of a process
taskset -p "$PID"

# Set affinity to a single CPU
taskset -p -c 0 "$PID"

# Set affinity to multiple CPUs (mask)
taskset -p 0x3 "$PID"    # CPUs 0 and 1

# Launch with affinity
taskset -c 0,2,4 command

Mask format: bit N = CPU N. 0x3 = CPUs 0 and 1. 0xf0 = CPUs 4, 5, 6, 7.

When to use affinity

• Latency-sensitive: pin to a single CPU with isolated IRQs.
• NUMA-aware: pin to a NUMA node.
• Cache-friendly: pin processes that share data to the same CPU.

For most workloads, the default scheduler is correct. Affinity is for specific performance problems.

NUMA topology

Modern multi-socket servers have multiple NUMA nodes:

numactl --hardware

Output:

available: 2 nodes (0-1)
node 0 cpus: 0 1 2 3 8 9 10 11
node 0 size: 64380 MB
node 0 free: 32000 MB
node 1 cpus: 4 5 6 7 12 13 14 15
node 1 size: 64400 MB
node 1 free: 30000 MB
node distances:
node   0   1
  0:  10  20
  1:  20  10

CPUs 0-3, 8-11 are on node 0. CPUs 4-7, 12-15 are on node 1. Local memory access is 10; remote is 20 (twice as slow).

numactl

# Run with NUMA-aware memory allocation
numactl --membind=0 command           # bind memory to node 0
numactl --cpunodebind=0 command       # bind CPU to node 0
numactl --preferred=0 command        # prefer node 0, fall back if needed

# Show current
numactl --show

For NUMA-sensitive workloads (large memory, high throughput), numactl can give 10-30% performance.

Per-process NUMA

# Substitute your own values before running:
PID=1234

# Pin memory of a process to node 0
numactl --membind=0 "$PID"

# Check current
cat /proc/"$PID"/numa_maps

The kernel’s numabalancing (auto-NUMA balancing) tries to migrate memory to the local node. For latency-sensitive workloads, manual pinning may be needed.

Systemd and NUMA

systemd can start services with NUMA pinning:

[Service]
ExecStart=/usr/local/bin/myapp
NUMAPolicy=bind
NUMAMask=0

This binds the service to NUMA node 0.

Common patterns

Knowledge check