Storage fundamentals: block, file, object

Why this matters in production

Storage is the most common performance and availability bottleneck. Picking the wrong storage class for a workload produces a system that is technically correct but operationally miserable. This lesson introduces the three primitives so the rest of the storage lessons have a common vocabulary.

The three primitives

Block storage

A virtual disk the guest treats as a raw block device. The guest owns the filesystem; the hypervisor just provides durable storage underneath.

Examples: VM disks on LVM, ZFS zvols, Ceph RBD, iSCSI LUNs.

flowchart LR
  G[Guest filesystem: ext4 / xfs / ntfs] --> D[Virtual disk]
  D --> H[Hypervisor block layer]
  H --> S[Physical disk]

Characteristics:

• Random-access reads and writes at block granularity.
• Best performance for VM workloads.
• Snapshots are filesystem-agnostic (a snapshot is just “don’t deallocate these blocks”).
• Storage stack decides snapshot semantics (ZFS, Ceph, LVM, file-backed qcow2).

File storage

A filesystem exposed over a network protocol (NFS, SMB). The guest or application sees a directory tree. Multiple clients can share the same filesystem.

Examples: NFS mount used for VM disks; SMB share for file server VMs.

flowchart LR
  A[Client] --> N[NFS / SMB protocol]
  N --> S[Server filesystem]

Characteristics:

• Good for shared data (file server VMs, ISO storage, templates).
• Locking semantics are protocol-dependent; NFS is “best effort” concurrent.
• Performance depends on the protocol and the server’s ability to handle concurrency.

Object storage

A flat namespace of objects addressed by key. Designed for massive scale-out. Examples: S3, Ceph RGW, MinIO.

Examples: PBS S3 datastores (where supported), archive storage.

flowchart LR
  A[Application] --> H[HTTP API]
  H --> O[Object store]

Characteristics:

• Virtually unlimited scale (no single namespace bottleneck).
• Eventual consistency in many implementations; not a database substitute.
• Best for archival, backups, large blobs.

How Proxmox uses each

The local-vs-shared-vs-distributed axis

Independent of the access primitive, storage can be:

These axes combine: block storage can be local (LVM on local disk) or distributed (Ceph RBD). File storage is usually shared (NFS). Object storage is usually distributed.

Choosing for a workload

Hands-on lab (Mode C)

For a given workload list, classify each VM’s storage requirement:

1. What primitive (block/file/object)?
2. What topology (local/shared/distributed)?
3. What performance class (low/medium/high IOPS)?

A 4-vCPU app server with 200 IOPS belongs on local block. A 40-VM cluster with replication needs distributed block (Ceph). A 200 TB archive needs object storage.

Common mistakes

• Using NFS for high-IOPS VM disks. NFS adds latency and contention; Ceph RBD or local ZFS is faster.
• Treating object storage as a database. It is not.
• Putting all VM disks on a single shared LUN; if the LUN fails, every VM is down.

Key takeaways

• Block, file, and object are the three primitives.
• VM disks are almost always block.
• Local, shared, and distributed are independent choices from the primitive.

Knowledge check