SATA SSD — the general-purpose Ceph OSD

Why this matters in production

SATA SSD is where most Ceph clusters that serve mixed workloads actually land. It removes the single worst property of HDD — the seek — at a price that still allows a lot of capacity, and its limitations are predictable enough to design around.

What changes versus HDD

There is no head to move, so random and sequential access cost approximately the same:

random read     ~0.1-0.2 ms   →  tens of thousands of IOPS
sequential read ~500 MB/s      (SATA 6 Gb/s interface limit)
random write    ~0.2-0.5 ms   →  thousands to tens of thousands

The gap against HDD is roughly two orders of magnitude on random work and modest on sequential. That asymmetry is exactly why an HDD cluster struggling with VM disks is transformed by SSDs while an HDD cluster serving an archive gains very little.

The two ceilings

Interface bandwidth. SATA 6 Gb/s tops out near 550 MB/s per device regardless of what the flash could do. For a Ceph OSD this is usually acceptable — a single OSD rarely needs more sustained bandwidth than that — but it caps large sequential work and it is the first thing NVMe removes.

Queue depth. SATA supports a queue depth of 32 through NCQ. NVMe supports thousands across many queues. Under heavy concurrency a SATA device saturates its queue and latency climbs, which matters most during recovery when a lot of parallel work arrives at once.

Consumer versus datacentre, in Ceph terms

The “sustained write” row is the one that surprises people benchmarking. A consumer drive has a fast SLC cache that absorbs the first tens of gigabytes at NVMe-like speed, then falls to the native TLC or QLC rate — sometimes below HDD. A Ceph OSD under recovery writes continuously for hours, which is exactly the case the cache cannot cover.

Choosing SATA SSD or NVMe

Reach for SATA SSD when capacity per pound matters, the workload is mixed, and 500 MB/s per device is sufficient. It is the sensible default for a general RBD pool serving VMs.

Reach for NVMe when latency is the requirement, when the device will carry BlueStore WAL and DB for several HDDs, when concurrency is high enough to saturate a queue depth of 32, or when the pool serves metadata — RGW indexes, CephFS metadata.

A very common and effective layout is a mixed cluster: NVMe for metadata pools and WAL/DB, SATA SSD for general RBD, HDD for archive and RGW data — three device classes, three CRUSH rules, one cluster.

Quiz

Production discipline

Require power-loss protection on every device that will hold OSD data, and treat its absence as disqualifying rather than as a trade-off. Specify sustained write rate rather than headline sequential figures, because a Ceph OSD under recovery writes for hours and never stays in a cache-friendly regime. Beyond that, SATA SSD is a sound default for general RBD, and the common production shape is a mixed cluster: NVMe for metadata and WAL/DB, SATA SSD for VM pools, HDD for archive.

Cross-course references

• Ceph: Part III lesson on enterprise SSDs develops power-loss protection.
• Ceph: Part XVI (Device Classes) for separating the tiers with CRUSH rules.
• Linux: Part XLI (Disk Performance) for benchmarking methodology that exhausts caches.