k and m: what an erasure code profile means

Why this matters in production

Erasure coding is how you store a petabyte without buying three. The arithmetic is simple; the operational consequences are not, and the single most common surprise is that a 4+2 pool does not keep serving after two failures the way people expect.

The mechanism

Each object written to an EC pool is split into k data chunks, from which m coding chunks are computed. All k+m chunks are stored on distinct OSDs.

object (4 MB), profile k=4 m=2
    ↓ split
  D1  D2  D3  D4          (1 MB each)
    ↓ compute
  D1  D2  D3  D4  C1  C2  (6 chunks, 6 OSDs)

Any k of the k+m chunks reconstruct the original object. That is the whole property: you can lose any m chunks and still recover the data.

Creating one

ceph osd erasure-code-profile set ec42 \
    k=4 m=2 \
    crush-failure-domain=host \
    plugin=jerasure technique=reed_sol_van

ceph osd pool create s3-archive erasure ec42
ceph osd pool get s3-archive erasure_code_profile

Inspect what you made:

ceph osd erasure-code-profile get ec42
ceph osd pool get s3-archive size min_size
# size: 6
# min_size: 5

The min_size surprise

Note that size reports 6 (= k+m) and min_size reports 5, not 4.

Data is recoverable from any 4 chunks. But Ceph sets min_size to k+1 by default, so the pool stops serving once fewer than 5 chunks are available — that is, after the second chunk loss, even though the data is still perfectly reconstructible.

Stated plainly for k=4, m=2:

The margin between min_size and k is deliberate. At exactly k chunks there is zero redundancy: any further loss during recovery is unrecoverable, and a write accepted at that point cannot be verified against a parity copy. k+1 keeps one chunk of margin, exactly as min_size 2 on a size-3 replicated pool does.

Reading the tolerance correctly

The honest statement for a k=4, m=2 pool is: tolerates 2 chunk losses without data loss, and 1 chunk loss without losing availability.

If you need to survive two concurrent failure-domain losses with I/O continuing, you need m=3.

Quiz

Production discipline

Write the availability property of every EC pool in terms of failure domains, not chunks — “keeps serving through one host loss” is what a stakeholder can act on. Verify min_size explicitly after creating an EC pool; it is the number that determines availability, and it is not the one people quote when they name a profile.

Cross-course references

• Kubernetes: this is the same arithmetic as tolerating node loss in a quorum-based StatefulSet
• Linux: RAID 6 is k+2 erasure coding with a fixed stripe layout