CephXXVI · Erasure Coding Trade-offsErasure Coding Trade-offs
EC durability compared with replication
What you'll learn
- Compare failure tolerance between EC profiles and replication
- Account for recovery window in the durability comparison
- Evaluate a profile against correlated failure
- State a durability claim that survives scrutiny
Prerequisites
None — start here.
Verified against Ceph Tentacle 20.2.x · Ceph Squid 19.2.x (supported previous) · cephadm matches the verified Ceph release · podman 4.x · csi-rbd and csi-cephfs current · RBD / CephFS / RGW current (matches Ceph release) · Linux kernel 5.15+ (5.10 minimum) · Ubuntu 24.04 LTS (Ceph host baseline) · Debian 12 (Bookworm) (Ceph host baseline) · Rocky Linux / RHEL / AlmaLinux 9.x (Ceph host baseline) · Proxmox VE 9.x (cross-course integration) · Kubernetes 1.31+ (cross-course integration) · 2026-08-18
Why this matters in production
“EC is less durable than replication” and “EC is more durable than replication” are both said confidently and neither is generally true. The comparison depends on the profile, the failure domain count, and the recovery window — and getting it right changes real design decisions.
Failure tolerance, side by side
| Configuration | Losses without data loss | Losses while serving | Raw cost |
|---|---|---|---|
| replicated size 3 | 2 | 1 | 3.0× |
| replicated size 4 | 3 | 2 | 4.0× |
| EC 4+2 | 2 | 1 | 1.5× |
| EC 6+3 | 3 | 2 | 1.5× |
| EC 8+3 | 3 | 2 | 1.375× |
| EC 10+4 | 4 | 3 | 1.4× |
On the raw tolerance column, EC 8+3 beats size 3 — three losses versus two — at less than half the capacity cost. That is a real advantage and it is why large archives use EC.
Where the comparison gets complicated
Recovery window. EC recovery moves k× the data. An 8+3 pool tolerates one more failure but stays exposed several times longer while rebuilding. Durability depends on failures during the window, so a longer window partially offsets the higher tolerance.
Chunk count and correlated failure. A size-3 PG occupies 3 failure domains; an 8+3 PG occupies 11. Every PG on a wide profile touches a much larger share of the cluster, so a correlated event affecting several domains is more likely to hit any given PG.
Domain count relative to k+m. A cluster with exactly k+m domains cannot self-heal at all after a domain failure. Replication on the same cluster, needing only 3 domains, heals normally.
Comparing honestly
State three things together:
- Losses tolerated without data loss
- Losses tolerated while still serving (
m − 1for EC,size − min_sizefor replicated) - Time to return to full redundancy after a single failure
An 8+3 pool on a 20-host cluster with 8 TB drives and a 100 Gb network is genuinely more durable than size 3. The same profile on a 12-host cluster with 20 TB drives and 10 Gb is not, despite identical k and m.
Quiz
Knowledge check · 4 questions
Q1. How does EC 8+3 compare with replicated size 3 on raw failure tolerance and capacity cost?
Q2. A wide EC profile spreads each PG across more failure domains, which increases exposure to correlated multi-domain failures.
Q3. Assess a durability claim in a design review.
A proposal claims an EC 10+4 pool is "more durable than three-way replication." The cluster has exactly 14 hosts in a single room on two power feeds, uses 20 TB drives, and has a 10 Gb cluster network.
Q4. Why can the same EC profile be the right choice on one cluster and the wrong one on another with identical k and m?
Passing score: 75%. Answers are checked in this browser.
Production discipline
Never state an EC durability claim as tolerance alone. Write it as tolerance, serving tolerance, and measured recovery window together, on the specific cluster — that triple is what a reviewer can check and what stays true as the cluster changes. Re-examine it after every hardware refresh, because larger drives quietly lengthen the window.
Cross-course references
- Kubernetes: quorum size and failure-domain spread interact the same way for stateful workloads
- Linux: RAID 6 versus RAID 10 durability arguments turn on the identical rebuild-window point