DAS, SAN, NAS — topology is not the same as primitive

Why this matters in production

Storage vocabulary mixes two independent axes and almost nobody says which one they mean. “We need a SAN” is a sentence about topology. “We need block storage” is a sentence about the primitive. They are routinely used as synonyms, and the confusion has consequences: it is why experienced storage engineers arrive at a Ceph cluster expecting multipath, LUN masking, and a fabric, and why their first incident is diagnosed against the wrong mental model.

Getting the two axes apart is a five-minute exercise that prevents years of miscommunication.

Two axes, not one

The primitive is what the storage does: block, file, or object. Covered in the preceding lessons.

The topology is how the consumer reaches it:

• DAS — Direct Attached Storage. The disk is on the machine’s own bus: SATA, SAS, NVMe over PCIe. No network. One consumer.
• SAN — Storage Area Network. Block devices reached over a dedicated storage fabric: Fibre Channel, iSCSI, FCoE, NVMe-oF. The fabric is its own thing, with its own switches, zoning, and multipath.
• NAS — Network Attached Storage. A filesystem reached over a general network with a file protocol: NFS, SMB.

flowchart TD
  subgraph Topology["How you reach it"]
    DAS[DAS<br/>local bus]
    SAN[SAN<br/>storage fabric]
    NAS[NAS<br/>file protocol]
    CEPH[Ceph<br/>IP network, native clients]
  end
  subgraph Primitive["What it does"]
    B[Block]
    F[File]
    O[Object]
  end
  DAS --> B
  SAN --> B
  NAS --> F
  CEPH --> B
  CEPH --> F
  CEPH --> O

Notice that the topology row constrains the primitive row but does not determine it. A SAN serves block. A NAS serves file. DAS serves block. Ceph is the one that serves all three, over an ordinary IP network, using clients that speak its own protocol.

Why Ceph is not a SAN

This is the comparison that causes real trouble, because RBD and a SAN LUN look identical from inside the VM. The differences are all below that line:

The practical consequences:

• There is no multipath to configure. A SAN engineer’s instinct is to check paths when latency rises. In Ceph the equivalent question is which OSD is slow, and the tool is ceph osd perf, not multipath -ll.
• There is no controller to fail over. There is a MON quorum to maintain, and losing it stops the cluster in a way no SAN controller failure does.
• Capacity growth is incremental and self-balancing, which is genuinely better — and it means adding a node triggers hours of backfill that a SAN expansion would not.

Where NAS still wins

Honesty about the alternatives is part of the design conversation. A well-run NFS server on a single box is simpler than a Ceph cluster, has lower latency for small metadata operations, and needs one machine’s worth of operational attention. If the requirement is “twelve hosts share 8 TiB and it must be simple”, CephFS is a large answer to a small question.

Ceph earns its complexity when you need one or more of: capacity that outgrows a single chassis, failure tolerance without a second array, several primitives from one pool of hardware, or growth without forklift upgrades. If none of those apply, the honest recommendation may be a NAS.

Quiz

Production discipline

Say which axis you mean. When someone asks for a SAN, establish whether they need block storage or a fabric, because Ceph gives one and not the other. Carry the translation table into incidents: multipath becomes CRUSH placement, controller failover becomes MON quorum, LUN becomes image, RAID rebuild becomes recovery. And hold the single-writer rule for RBD absolutely — the SAN instinct to present one volume to two hosts is the one imported habit that destroys filesystems.

Cross-course references

• Linux: Part XIII (Disks and Block Devices) and Part XVII (RAID) for the local and array-side comparisons.
• Proxmox: Part VII (Shared Storage) covers the NAS and SAN options alongside Ceph.
• VyOS: the network design parts are directly relevant, since Ceph replaces a fabric with your IP network.