CephLXXX · Blocked OperationsBlocked Operations
Blocked and slow: the distinction that changes everything
What you'll learn
- Distinguish blocked from slow definitively
- Explain why the responses differ
- Apply the right first checks to each
- Communicate the distinction to stakeholders
Prerequisites
None — start here.
Verified against Ceph Tentacle 20.2.x · Ceph Squid 19.2.x (supported previous) · cephadm matches the verified Ceph release · podman 4.x · csi-rbd and csi-cephfs current · RBD / CephFS / RGW current (matches Ceph release) · Linux kernel 5.15+ (5.10 minimum) · Ubuntu 24.04 LTS (Ceph host baseline) · Debian 12 (Bookworm) (Ceph host baseline) · Rocky Linux / RHEL / AlmaLinux 9.x (Ceph host baseline) · Proxmox VE 9.x (cross-course integration) · Kubernetes 1.31+ (cross-course integration) · 2026-08-18
Why this matters in production
Slow operations complete and blocked ones do not. Performance work on a blocked condition achieves nothing, and structural investigation of a slow one wastes the window in which tuning would have helped.
The definitive test
ceph health detail | grep -oE '[0-9]+ (slow ops|ops).*blocked for [0-9]+'
sleep 120
ceph health detail | grep -oE '[0-9]+ (slow ops|ops).*blocked for [0-9]+'
| Observation | Condition |
|---|---|
| Count falls, age bounded | slow — operations completing |
| Count static, age grows by the elapsed time | blocked — nothing completing |
| Count grows, age bounded | overload |
| Count grows, age grows | worsening; both |
Two minutes and two commands settle it.
Why the responses differ
Slow:
cause is contention, a device, a network path
the cluster is functioning and constrained
remedies: tuning, throttling, replacing hardware
Blocked:
cause is a state that forbids progress
the cluster is functioning correctly and refusing
remedies: change the state
Tuning a blocked condition is the common error: throttles are adjusted, concurrency is changed, and nothing moves because nothing was ever going to.
The first checks for each
# slow — performance layers
IFACE=bond0
POOL=rbd-vms
ceph osd perf | sort -k2 -rn | head -5
iostat -x 1 5
ethtool -S ${IFACE} | grep -i err
# blocked — structural conditions
ceph -s
ceph health detail
ceph osd dump | grep flags
ceph osd pool get-quota ${POOL}
ceph quorum_status
The blocked checks are all cheap state lookups, which is why the sweep takes under a minute.
Communicating the distinction
| To stakeholders | Slow | Blocked |
|---|---|---|
| What is happening | requests are taking longer | requests are not completing |
| Impact | degraded performance | effective outage for affected data |
| Expected resolution | after tuning or hardware work | once a specific condition is changed |
| Can it self-resolve | often | rarely |
Requests to this pool are not completing rather than running slowly. The cluster is refusing them because a condition prevents progress; we have identified it as [X] and are changing it. Performance tuning would not have helped.
That framing correctly sets expectations in both directions.
Quiz
Knowledge check · 4 questions
Q1. Two samples two minutes apart show the same operation count with the oldest age up by 120 seconds. What does this indicate?
Q2. Twenty minutes of tuning that changes nothing is itself evidence about which condition you are facing.
Q3. Respond to operations not completing.
Operations against a pool are not completing. The team has spent twenty minutes adjusting mClock profiles and backfill settings with no effect.
Q4. How should a blocked condition be described to stakeholders?
Passing score: 75%. Answers are checked in this browser.
Production discipline
Sample the operation count and oldest age twice before choosing a response path — the age growing by exactly the elapsed time with a static count is the definition of blocked, not a proxy for it. Run the structural sweep rather than tuning; it takes under a minute and tuning cannot help.
Cross-course references
- Kubernetes: a Pending pod and a slow pod need entirely different investigations
- Linux: a process in uninterruptible sleep is not a performance problem