CephFS as Proxmox storage

Intermediate⏱ ~17 mincephpvesmmount

What you'll learn

Configure CephFS storage in Proxmox VE
Choose between CephFS and RBD per content type
Understand the sharing semantics
Diagnose CephFS mount problems

Prerequisites

None — start here.

Verified against Ceph Tentacle 20.2.x · Ceph Squid 19.2.x (supported previous) · cephadm matches the verified Ceph release · podman 4.x · csi-rbd and csi-cephfs current · RBD / CephFS / RGW current (matches Ceph release) · Linux kernel 5.15+ (5.10 minimum) · Ubuntu 24.04 LTS (Ceph host baseline) · Debian 12 (Bookworm) (Ceph host baseline) · Rocky Linux / RHEL / AlmaLinux 9.x (Ceph host baseline) · Proxmox VE 9.x (cross-course integration) · Kubernetes 1.31+ (cross-course integration) · 2026-08-18

Why this matters in production

RBD gives each VM its own block device; CephFS gives every node the same filesystem. The distinction determines which content types belong where.

Configuring it

# on the Ceph side
ceph fs volume create pve-fs
ceph auth get-or-create client.pve-fs \
  mon 'allow r' \
  mds 'allow rw' \
  osd 'allow rw tag cephfs data=pve-fs'

# on the Proxmox side
pvesm add cephfs ceph-shared \
  --monhost '10.0.0.11,10.0.0.12,10.0.0.13' \
  --username pve-fs \
  --content backup,iso,vztmpl,snippets

# /etc/pve/storage.cfg
cephfs: ceph-shared
        monhost 10.0.0.11,10.0.0.12,10.0.0.13
        username pve-fs
        content backup,iso,vztmpl,snippets
        fs-name pve-fs

Choosing per content type

Content	Storage	Why
VM disk images	RBD	block semantics, per-VM isolation
Container rootfs	RBD with `krbd 1`	block device required
ISO images	CephFS	shared, read by every node
Container templates	CephFS	same
Backups	CephFS	shared, large sequential
Snippets and hook scripts	CephFS	shared configuration

The pattern: anything every node needs to read is CephFS; anything one VM owns is RBD.

CephFS: every Proxmox node mounts the same filesystem
        a file written on node A is immediately visible on node B
        POSIX semantics including locking

This is what makes backups and ISO storage work: a backup taken on one node is restorable on any other, and an ISO uploaded once is available everywhere.

mount | grep ceph
df -h /mnt/pve/ceph-shared

Proxmox mounts CephFS storages under /mnt/pve/<storage-id> on every node.

Diagnosing mount problems

pvesm status
systemctl status mnt-pve-ceph\\x2dshared.mount
journalctl -u mnt-pve-ceph\\x2dshared.mount --since '1 hour ago'
dmesg -T | grep -i ceph | tail -20

Symptom	Cause
Mount fails, permission denied	MDS or OSD capability mismatch
Mount hangs	monitors unreachable, or MDS unavailable
Storage inactive on one node only	that node’s network or keyring
Slow directory operations	MDS overloaded, or metadata pool on HDD
`stale file handle`	client blocklisted after a network interruption

ceph fs status pve-fs
ceph mds stat
ceph osd blocklist ls

Quiz

Knowledge check · 4 questions

Q1. Which content types belong on CephFS rather than RBD in a Proxmox deployment?

VM disk images and container rootfs, since both are large files every node must be able to open Only backups, as they are the one content type Proxmox writes from more than a single node Everything, since CephFS offers POSIX semantics that cover block workloads without further configuration ISO images, container templates, backups, and snippets — anything every node needs to read
Q2. A blocklisted CephFS client reconnects automatically once the network recovers.

True False
Q3. Design storage layout for a Proxmox cluster.

A four-node Proxmox cluster backed by Ceph needs storage for VM disks, container rootfs, ISO images, templates, and backups.
Q4. Why does Proxmox performance benefit disproportionately from a fast CephFS metadata pool?

Passing score: 75%. Answers are checked in this browser.

Production discipline

Split by access pattern: RBD for anything one VM owns, CephFS for anything every node reads. Put the CephFS metadata pool on NVMe — Proxmox performs many metadata operations and their latency is what the interface feels like.

Cross-course references

Kubernetes: ReadWriteMany versus ReadWriteOnce volumes is the same distinction
Linux: shared filesystem versus block device selection follows identical reasoning