CephLVI · OSD FailureOSD Failure
What degraded PGs mean after an OSD failure
What you'll learn
- Interpret PG states following an OSD failure
- Assess actual exposure from the state and counts
- Distinguish degraded from undersized and incomplete
- Decide urgency from the PG picture
Prerequisites
None — start here.
Verified against Ceph Tentacle 20.2.x · Ceph Squid 19.2.x (supported previous) · cephadm matches the verified Ceph release · podman 4.x · csi-rbd and csi-cephfs current · RBD / CephFS / RGW current (matches Ceph release) · Linux kernel 5.15+ (5.10 minimum) · Ubuntu 24.04 LTS (Ceph host baseline) · Debian 12 (Bookworm) (Ceph host baseline) · Rocky Linux / RHEL / AlmaLinux 9.x (Ceph host baseline) · Proxmox VE 9.x (cross-course integration) · Kubernetes 1.31+ (cross-course integration) · 2026-08-18
Why this matters in production
The PG states after a failure tell you exactly how exposed you are, and the words are precise. Degraded and undersized mean different things, and incomplete means something much worse.
The states
ceph pg stat
ceph health detail
ceph pg dump_stuck| State | Means | Serving? |
|---|---|---|
active+clean | full complement, all current | yes |
active+undersized | fewer OSDs than size in the acting set | yes, above min_size |
active+degraded | some objects lack full copies | yes |
active+undersized+degraded | both, the usual state after a failure | yes |
active+recovering+degraded | actively restoring copies | yes |
active+remapped+backfilling | copies moving to new locations | yes |
undersized+degraded+incomplete | below min_size | no |
down | peering cannot proceed | no |
The first six all serve. The last two do not, and that is the distinction that determines urgency.
Degraded versus undersized
Undersized is about the acting set: fewer OSDs than the pool’s size.
Degraded is about the objects: some do not have the full number of copies.
They usually appear together after a failure, but not always. A PG can be undersized without being degraded if no writes have occurred since the OSD left — the remaining copies are complete and current, there are simply fewer of them than there should be.
Assessing exposure
ceph health detail | grep -A3 PG_DEGRADED[WRN] PG_DEGRADED: Degraded data redundancy: 41213/18442104 objects
degraded (0.223%), 102 pgs degraded, 102 pgs undersizedThree figures matter: the object count, the percentage, and the PG count.
| Reading | Exposure |
|---|---|
| Small percentage, size 3 pool | two copies remain; routine |
| Large percentage, size 3 pool | two copies across much of the cluster |
| Any PGs incomplete or down | clients blocked |
| Degraded on a size 2 pool | one copy remains; urgent |
Checking for the serious cases
ceph pg dump_stuck inactive
ceph health detail | grep -E 'PG_AVAILABILITY|incomplete|down'If this returns nothing, the failure is being handled and the response is to let recovery run. If it returns PGs, clients are blocked and the urgency changes completely.
Quiz
Knowledge check · 4 questions
Q1. A PG reports `undersized+degraded+incomplete`. What is the client impact?
Q2. A PG is always degraded when it is undersized.
Q3. Assess exposure after a failure.
An OSD failed. `ceph health detail` reports 0.02% of objects degraded across 47 PGs, all in the `active+undersized+degraded` state. The pool is size 3, min_size 2. Recovery is progressing.
Q4. Which single check distinguishes a routine failure from a client-affecting one?
Passing score: 75%. Answers are checked in this browser.
Production discipline
Check for inactive PGs before assessing degradation; the two situations look similar in a health summary and have completely different urgency. Read the object percentage alongside the PG count — the same number of degraded PGs can represent a trivial or a substantial share of the data.
Cross-course references
- Kubernetes: distinguishing degraded replicas from unavailable ones drives the same urgency split
- Linux: a RAID array degraded versus failed is the identical distinction