Large immutable data: the clearest EC case

Intermediate⏱ ~16 minceph

What you'll learn

Identify the shared characteristics of write-once workloads
Match profile width to the immutability of the data
Recognise near-immutable workloads that still fit
Design retention-aware EC storage

Prerequisites

None — start here.

Verified against Ceph Tentacle 20.2.x · Ceph Squid 19.2.x (supported previous) · cephadm matches the verified Ceph release · podman 4.x · csi-rbd and csi-cephfs current · RBD / CephFS / RGW current (matches Ceph release) · Linux kernel 5.15+ (5.10 minimum) · Ubuntu 24.04 LTS (Ceph host baseline) · Debian 12 (Bookworm) (Ceph host baseline) · Rocky Linux / RHEL / AlmaLinux 9.x (Ceph host baseline) · Proxmox VE 9.x (cross-course integration) · Kubernetes 1.31+ (cross-course integration) · 2026-08-18

Why this matters in production

Large immutable data is usually the biggest single line in a storage budget and the easiest place to halve it. It is also where the widest EC profiles are safe to use, because the properties that make wide profiles painful — small writes, latency sensitivity, frequent modification — are all absent.

The shared profile

Property	Backups	Log archives	Media	Scientific data
Written	once	append-only	once	once
Modified	never	never	rarely	never
Read	rarely	occasionally	occasionally	in bulk
Object size	large	large after batching	large	very large
Latency need	low	low	moderate	low
Volume	very large	large	very large	very large

Every row points the same direction. These workloads have no read-modify-write, no latency budget to blow, and enough volume that the capacity saving is measured in racks.

Profile width and immutability

Wide profiles are penalised by write amplification, latency, and degraded-read cost. Immutable data pays none of the first two and rarely encounters the third:

Data	Suggested	Reasoning
Nightly backups, 30-day retention	8+3	good ratio, manageable recovery
Long-term archive, 7-year retention	10+4 or wider	capacity dominates; reads are rare
Media served to users	6+3 or 8+3	reads matter; keep recovery moderate
Active scientific datasets	6+3	bulk reads are frequent

The moderating factor is read frequency during degraded operation. Data that is genuinely cold can take the widest profile the cluster’s domain count supports; data that is read regularly should stay narrower so degraded reads do not become a visible problem.

Near-immutable workloads

Some workloads are not strictly write-once but behave that way:

Time-series data past its retention edge — written once, then only aged out
Container image layers — immutable by content-address
Compliance archives with WORM semantics
Build artefacts — written by CI, never modified

Deletion is not modification. An EC pool handles object deletion cleanly; it is in-place overwrite that costs.

Quiz

Knowledge check · 4 questions

Q1. Why can genuinely cold archival data use a wider EC profile than actively-read data?

Cold data compresses better, so a wide profile stores more logical bytes behind each coding chunk Wide profiles only reach their advertised capacity overhead on data that is never read back It rarely encounters degraded reads, which are the main cost of a wide profile at read time Cold data can be placed without a failure domain rule, so k+m stops constraining the layout
Q2. Deleting objects from an EC pool requires reading and recomputing coding chunks.

True False
Q3. Make a time-series archive suitable for EC.

A metrics platform writes 60 million data points per minute directly to a Ceph pool as small individual objects, and retains them for two years. The team wants to move it to EC to control the 400 TB and growing footprint.
Q4. Why is deletion cheap on an EC pool while in-place overwrite is expensive?

Passing score: 75%. Answers are checked in this browser.

Production discipline

When evaluating a workload for EC, ask about object size at the storage layer rather than at the application layer — the two often differ, and batching at ingest is frequently the cheapest change available. Record the read-frequency assumption behind any wide profile choice, since it is what justifies accepting the degraded-read cost.

Cross-course references

Kubernetes: CronJob-driven batch compaction before archival is the usual place this batching happens
Linux: log rotation into compressed archives is the same transformation at a smaller scale