The cost of erasure coding

Why this matters in production

EC’s headline is that it stores more data per disk. The ratio is easy to compute and the costs it omits are what determine whether a profile is right.

The arithmetic

raw per logical byte = (k + m) / k
usable fraction      = k / (k + m)
tolerates            = m failures

ceph osd erasure-code-profile get ec42
ceph osd pool get ec-pool erasure_code_profile
ceph df detail

Note the first three rows: the same 1.5× overhead buys one, two, or three failure tolerance depending on the profile. Larger k and m at the same ratio is strictly better on durability — and worse on everything else.

What the ratio does not show

The host count constraint is the one that most often rules out a profile: a 6-host cluster cannot run k=8 m=3 with failure_domain=host at all.

Small object overhead

object size 8 KiB, k=4, stripe unit 4 KiB
  → split into 4 chunks of 2 KiB each
  → padded to the stripe unit → 4 KiB each
  → 4 data + 2 parity = 6 × 4 KiB = 24 KiB raw for an 8 KiB object
  → effective multiplier 3×, not 1.5×

ceph df detail
# compare STORED against USED for the EC pool

If USED / STORED is much higher than (k+m)/k, small objects are the reason.

Choosing a profile

Quiz

Production discipline

Verify an EC pool’s real overhead with USED / STORED in ceph df detail rather than the profile’s theoretical ratio; small objects padded to the stripe unit can push it to replication levels. Check that k+m fits the failure domain before choosing a profile — it cannot be changed after the pool exists.

Cross-course references

• Kubernetes: quorum size versus tolerance is the same scaling trade-off
• Linux: RAID6 versus RAID5 presents the analogous parity-count decision