LinuxXXXVIII · Linux Performance Fundamentalsmpstat pidstat
mpstat and pidstat - per-CPU and per-process statistics
What you'll learn
- Use mpstat for per-CPU statistics
- Use pidstat for per-process statistics
- Identify context-switch and CPU saturation
- Spot single-thread bottlenecks
Prerequisites
Verified against Ubuntu 24.04 LTS · Debian 12 (Bookworm) · RHEL 9.x · Rocky Linux 9.x · AlmaLinux 9.x · Linux kernel 6.1 LTS / 6.6 LTS · systemd 255+ · OpenSSH 8.7p1 (RHEL 9) / 9.6p1 (Ubuntu 24.04) · nftables 1.0.x · chrony 4.x · Pacemaker 2.1.x · Corosync 3.1.x · 2026-08-09
mpstat and pidstat (from sysstat) are the per-resource performance tools. mpstat breaks down by CPU; pidstat breaks down by process. They are the next step after top.
mpstat
mpstat 1
Output:
Linux 6.1.0 (host) 2026-08-09 _x86_64_ (8 CPU)
14:30:00 CPU %usr %nice %sys %iowait %irq %soft %steal %guest %idle
14:30:01 all 20.00 0.00 5.00 5.00 0.00 0.00 0.00 0.00 70.00
14:30:01 0 45.00 0.00 5.00 5.00 0.00 0.00 0.00 0.00 45.00
14:30:01 1 15.00 0.00 5.00 5.00 0.00 0.00 0.00 0.00 75.00
14:30:01 2 15.00 0.00 5.00 0.00 0.00 0.00 0.00 0.00 80.00
14:30:01 3 15.00 0.00 5.00 5.00 0.00 0.00 0.00 0.00 75.00
14:30:01 4 15.00 0.00 5.00 5.00 0.00 0.00 0.00 0.00 80.00
14:30:01 5 15.00 0.00 5.00 0.00 0.00 0.00 0.00 0.00 85.00
14:30:01 6 15.00 0.00 5.00 0.00 0.00 0.00 0.00 0.00 85.00
14:30:01 7 15.00 0.00 5.00 0.00 0.00 0.00 0.00 0.00 85.00
Reading:
all: aggregated across all CPUs.- CPU 0: 45% user, much higher than other CPUs. This is a single-threaded application bottleneck.
- CPU 0-3: similar %usr.
%iowait: low (0-5%). Not I/O bound.
Single-thread usage on CPU 0 indicates a multi-thread application has a bottleneck on one core. Often this is locking or poor thread scaling.
pidstat
pidstat 1
Output:
14:30:00 UID PID %usr %system %guest %wait %CPU CPU Command
14:30:01 1000 1234 45.00 2.00 0.00 0.10 47.00 0 java
14:30:01 1000 1235 2.00 0.50 0.00 0.05 2.50 1 java
14:30:01 1000 1236 2.00 0.50 0.00 0.05 2.50 2 java
14:30:01 1000 1237 2.00 0.50 0.00 0.05 2.50 3 java
...
Each PID’s CPU usage. The application is multi-threaded, but one thread is using 47% CPU while others are at 2.5%. This is a thread imbalance - likely a lock contention or a single-threaded section in the code.
Context switches
pidstat -w 1
Output:
14:30:00 UID PID cswch/s nvcswch/s Command
14:30:01 1000 1234 500.00 1.50 java
14:30:01 1000 1235 800.00 2.10 java
cswch/s: voluntary context switches (process waits for
I/O). nvcswch/s: involuntary (kernel preempts). High
nvcswch/s indicates CPU saturation or thread contention.
Memory per-process
pidstat -r 1
Output:
14:30:00 UID PID minflt/s majflt/s VSZ RSS %MEM Command
14:30:01 1000 1234 5.00 0.00 8G 2G 3.00 java
minflt/s: minor faults (no I/O). majflt/s: major faults
(disk I/O). High majflt/s indicates memory pressure.
I/O per-process
pidstat -d 1
Output shows per-process read/write rates. The process doing the most I/O is the one to investigate.
Common patterns
| Pattern | Cause |
|---|---|
| One CPU at 100%, others idle | Single-thread bottleneck, lock contention |
| All CPUs at 90% | CPU bound; check what is using CPU |
| High voluntary cswch | I/O wait; check disk/network |
| High involuntary cswch | CPU saturation or preemption |
| High minflt/s | Lots of allocation; check code |
| High majflt/s | Memory pressure; check RAM and swap |
Knowledge check
Knowledge check · 3 questions
Q1. What does pidstat -w show?
Q2. One CPU at 100% while others are idle indicates a single-thread bottleneck.
Q3. Which of the following are valid pidstat options? Select all that apply.
Passing score: 75%. Answers are checked in this browser.