Random and sequential, read and write — the four profiles

Why this matters in production

Access profile decides which hardware matters. A sequential-read workload is limited by bandwidth — network, then device. A random-read workload is limited by device latency and by how many spindles or channels you have. A random-write workload on Ceph is limited by the slowest replica. These are different bottlenecks, and buying the wrong hardware to fix the wrong one is expensive and common.

The four profiles

Sequential read. Consecutive offsets, predictable. Readahead works, caches work, spinning disks perform well. Backups, media streaming, table scans, Ceph recovery reads.

Sequential write. Consecutive offsets, appended. Write caching and coalescing work well. Log ingestion, backup targets, database WAL.

Random read. Scattered offsets, unpredictable. Readahead is useless and often harmful. Device latency dominates. Database index lookups, VM boot, metadata traversal.

Random write. Scattered offsets, scattered commits. The hardest profile on every device class, and the one Ceph amplifies most. Database page writes, VM disk activity, filesystem metadata.

How each device class responds

The interesting column is HDD. A spinning disk is entirely competitive on sequential work and roughly two orders of magnitude worse on random work, because every random operation costs a seek. That single fact is why HDD-backed Ceph clusters are excellent for archives and object storage and painful for VM disks — and why the answer is usually not “no HDDs” but “HDD for the right pool”.

The Ceph asymmetry

Reads and writes are not symmetric in a replicated pool:

flowchart LR
  subgraph Read
    C1[Client] -->|read| P1[Primary OSD]
    P1 -->|data| C1
  end
  subgraph Write
    C2[Client] -->|write| P2[Primary OSD]
    P2 --> R1[Replica 1]
    P2 --> R2[Replica 2]
    R1 -->|ack| P2
    R2 -->|ack| P2
    P2 -->|ack| C2
  end

A read is served by the primary OSD alone: one network hop, one device read. A write goes to the primary, is forwarded to every other member of the acting set, and is acknowledged only when all of them have committed. So one client write costs size device writes and its latency is that of the slowest replica.

On an erasure-coded pool the asymmetry is different and often worse for small writes, because a partial stripe write requires reading the other chunks to recompute parity.

Using the profile to predict the bottleneck

• Sequential, large, either direction → suspect network first, then device bandwidth. This is where a shared 10 GbE link shows up.
• Random read → suspect device latency and the number of OSDs the data is spread across.
• Random write → suspect replication factor, the slowest OSD, and BlueStore metadata placement, in that order.

That ordering saves hours. A random-write complaint on an HDD cluster with WAL and DB on the same spindles has a known answer before you open a single graph.

Quiz

Production discipline

Classify the workload before choosing hardware or diagnosing a complaint; the profile determines which resource is the ceiling. Plan HDD capacity in random IOPS rather than terabytes, and remember that replication multiplies every client write by size. When a random-write cluster is slow and every OSD is equally slow, stop looking for a failing device — the cluster is at its budget, and the answers are BlueStore metadata on NVMe, more spindles, or a different pool for the latency-sensitive workloads.

Cross-course references

• Linux: Part XLI (Disk Performance) for readahead and the block-layer view.
• Ceph: Part XII (BlueStore) for WAL and DB device placement.
• Ceph: Part XXIII (Replication) for the write path in detail.