Building the PBS host: install, storage layout, sizing · Proxmox VE

The rest of this part teaches PBS as something that already exists: you point a backup job at it, you prune it, you restore from it. This lesson is about building the thing. It comes late in the part because the design decisions only make sense once you know what a chunk store is — the hardware shape of a PBS host is a consequence of how the chunk store behaves, not a generic “backup server” spec.

Two of the decisions here cannot be changed later without rebuilding: the datastore filesystem, and whether the datastore has a metadata device. Both are made in the first hour and paid for over the following three years.

The separation rule

The Proxmox documentation is unambiguous, and it is worth quoting rather than paraphrasing:

Installing the backup server directly on the hypervisor is not recommended. It is safer to use a separate physical server to store backups.

The word doing the work there is physical. A PBS instance running as a VM on the cluster it protects fails the same way as a backup on the same disk as the data: the failure that destroys the source destroys the copy. The scenarios are not exotic:

Failure	PBS on the cluster	PBS on its own box
Ceph pool loses data	Datastore is on that pool. Backups gone.	Unaffected
Cluster loses quorum	PBS VM will not start; `/etc/pve` read-only	Unaffected, still reachable
Bad kernel or firmware takes every node down	Nothing to restore from	Restore source intact
An operator with root on PVE is compromised	Same credentials reach the datastore	Separate host, separate auth realm
You need to restore the cluster	Chicken and egg	Ordinary restore

The last row is the one that turns an inconvenience into an outage of indefinite length. Every disaster-recovery plan that starts “restore the guests from PBS” assumes PBS is a thing that survives the disaster.

Sizing, derived rather than guessed

The published recommendations are the floor, not the answer:

Resource	Documented recommendation
CPU	Modern 64-bit AMD or Intel, at least 4 cores
RAM	Minimum 4 GiB for OS, filesystem cache and daemons, plus at least 1 GiB per TiB of storage
OS storage	32 GiB or more, on hardware RAID with a battery-backed write cache, or a redundant ZFS setup
Backup storage	Fast local storage delivering high IOPS for random IO; enterprise SSDs for best results
Network	Redundant multi-Gbit/s NICs

The RAM formula is the one people misread. It is additive: a 40 TiB datastore wants 4 + 40 = 44 GiB, not 4 GiB. It scales with the datastore because the workload that dominates PBS is not sequential streaming — it is metadata lookup across a very large number of small files, and that only works at speed if the metadata is cached.

The arithmetic that actually determines capacity

Start from the inventory, not from a percentage.

Read-only / Safecapacity worked example - run the arithmetic, do not eyeball it— Everything here is derived from numbers you can measure on the source cluster before you buy anything.

set -euo pipefail

# --- inputs you must measure, not assume -----------------------------------
PROVISIONED_TIB=60      # sum of virtual disk sizes across all guests
USED_FRACTION=0.55      # how full those disks actually are (measure in-guest)
DAILY_CHANGE_PCT=2      # fraction of USED data rewritten per day
KEEP_DAILY=14
KEEP_WEEKLY=8
KEEP_MONTHLY=12
DEDUP_FACTOR=2.5        # placeholder ONLY until you have measured your own

# --- derived ----------------------------------------------------------------
USED_TIB=$(echo "$PROVISIONED_TIB * $USED_FRACTION" | bc -l)

# The first backup stores the used data once, after deduplication.
BASE_TIB=$(echo "$USED_TIB / $DEDUP_FACTOR" | bc -l)

# Every retained snapshot after the first adds only its unique chunks.
SNAPSHOTS=$(( KEEP_DAILY + KEEP_WEEKLY + KEEP_MONTHLY ))
DELTA_TIB=$(echo "$USED_TIB * $DAILY_CHANGE_PCT / 100 * $SNAPSHOTS" | bc -l)

RAW_TIB=$(echo "($BASE_TIB + $DELTA_TIB) * 1.30" | bc -l)   # 30% headroom

printf 'used            %.1f TiB\n' "$USED_TIB"
printf 'base after dedup %.1f TiB\n' "$BASE_TIB"
printf 'retained deltas  %.1f TiB over %d snapshots\n' "$DELTA_TIB" "$SNAPSHOTS"
printf 'buy at least     %.1f TiB usable\n' "$RAW_TIB"

Three things about that script matter more than the numbers it prints.

DEDUP_FACTOR is a placeholder and the script says so. Deduplication ratio is a property of your fleet, not of PBS. Fifty near-identical Debian VMs deduplicate spectacularly; fifty databases holding different customer data barely deduplicate at all. Plan with a conservative figure, then replace it with the real one after two weeks of backups — PBS reports it directly.

The 30% headroom is not padding. Garbage collection needs free space to work in, a datastore above roughly 80% starts allocating badly on any copy-on-write filesystem, and the single most common PBS incident is a datastore that filled up and stopped accepting the backups you were relying on.

Change rate dominates at long retention. At 2% daily change and 34 retained snapshots, the deltas are two thirds of a base copy. Doubling retention roughly doubles the delta term while leaving the base term untouched, which is why “keep everything for a year” is a capacity decision disguised as a policy decision.

Read-only / Safemeasure the real deduplication factor once you have data— Then read the per-datastore usage and deduplication figure from the datastore summary in the GUI, or from the API.

# proxmox-backup-manager datastore list --output-format json | jq '.[] | {name, path}'

[
{
  "name": "store1",
  "path": "/mnt/datastore/store1"
}
]

Illustrative output

IOPS: why the chunk store wants SSD or a metadata device

A PBS datastore is not a big file. It is a directory of millions of small ones. From the technical overview:

The chunks of a datastore are found in <datastore-root>/.chunks/ … These chunk directories (0000-ffff) will be preallocated when a datastore is created.

That is 65,536 directories created up front, into which chunks are filed by the first two bytes of their checksum. For block-based backups the chunks are fixed size, typically 4 MiB. So:

Referenced data	Chunks at 4 MiB	Chunks per directory
1 TiB	262,144	~4
10 TiB	2,621,440	~40
100 TiB	26,214,400	~400

Now consider garbage collection, which walks every index in the datastore and touches the access time of every chunk it references, then sweeps. On a 100 TiB datastore that is tens of millions of random metadata operations. On enterprise SSD it is a background job. On a spinning-disk pool with no metadata acceleration it is a job that runs for a day and competes with the backup window.

Laying out the disks

A workable three-tier layout for a dedicated PBS host:

Tier	Devices	Purpose
OS	2 × small SSD, ZFS mirror (or hardware RAID1 with BBU)	Debian 13 + PBS. 32 GiB minimum; 64+ GiB is more comfortable once task logs accumulate
Datastore data	HDD RAIDZ2 or SSD mirrors/RAID10	The `.chunks/` bulk
Datastore metadata	2 × NVMe mirrored, as a ZFS `special` vdev	Only if the data tier is spinning

Destructivecreating the datastore pool - these commands erase the named devices— Run only against devices you have confirmed are empty. Identify disks by /dev/disk/by-id, never by /dev/sdX, which is not stable across reboots.

set -euo pipefail

# Confirm what PBS thinks the disks are before you touch them.
proxmox-backup-manager disk list

# Option A: let PBS build the pool and register the datastore in one step.
proxmox-backup-manager disk zpool create bulk \
--devices /dev/disk/by-id/wwn-0x5000c500aaaa0001,/dev/disk/by-id/wwn-0x5000c500aaaa0002,/dev/disk/by-id/wwn-0x5000c500aaaa0003,/dev/disk/by-id/wwn-0x5000c500aaaa0004 \
--raidlevel raidz2 \
--add-datastore true

# Option B: build the pool by hand when you need a special vdev, which the
# PBS helper does not create for you.
zpool create -o ashift=12 bulk \
raidz2 /dev/disk/by-id/wwn-0x5000c500aaaa0001 \
       /dev/disk/by-id/wwn-0x5000c500aaaa0002 \
       /dev/disk/by-id/wwn-0x5000c500aaaa0003 \
       /dev/disk/by-id/wwn-0x5000c500aaaa0004 \
special mirror /dev/disk/by-id/nvme-eui.0001 /dev/disk/by-id/nvme-eui.0002

zfs set compression=zstd atime=on relatime=off bulk
zfs create bulk/store1

Creating the datastore, and the two tuning options

Configuration changedatastore creation and schedules— Creates the datastore, preallocates the 65,536 chunk directories, and attaches a GC and prune schedule.

set -euo pipefail

proxmox-backup-manager datastore create store1 /bulk/store1

# Schedules use systemd calendar-event syntax.
proxmox-backup-manager datastore update store1 \
--gc-schedule 'sat 03:00' \
--prune-schedule 'daily 01:00' \
--keep-daily 14 --keep-weekly 8 --keep-monthly 12

proxmox-backup-manager datastore list

Two tuning options exist and both are durability decisions rather than performance tweaks:

Option	Values	What it changes
`chunk-order`	`inode` (default), `none`	Whether verification and GC read chunks in filesystem-inode order or in index order. `inode` is dramatically better on spinning disks; on NVMe the difference is small
`sync-level`	`none`, `filesystem` (default), `file`	How aggressively PBS forces data to stable storage. `file` fsyncs each chunk; `filesystem` syncs at the end of a backup; `none` leaves it to the OS

Configuration changedatastore tuning— Set only with a reason you can write down. The defaults are correct for the majority of deployments.

# Spinning-disk datastore, power-protected: the default ordering, maximum
# throughput.
proxmox-backup-manager datastore update store1 --tuning 'chunk-order=inode'

# A datastore on hardware with no power protection, where a host crash must
# not be able to leave a backup that verifies but cannot be read.
proxmox-backup-manager datastore update store1 --tuning 'sync-level=file'

Why sync-level is a real choice and not just a slower setting

Every chunk PBS writes is content-addressed: its filename is the checksum of its contents. That property is what makes sync-level=none tempting — a corrupted chunk will fail verification, so surely it gets caught?

It does get caught, eventually, by a verify job. The gap is when. With sync-level=none the server acknowledges a backup as complete while chunks may still be in the page cache. If the host loses power in that window, the index references chunks that were never written. The backup appears in the snapshot list, the job reported success, the notification said green, and the data is not there. You find out at the next verify job — or at the restore.

sync-level=filesystem, the default, syncs when the backup finishes, so the acknowledgement means something. sync-level=file syncs per chunk and is the setting for hardware you do not trust, at a real throughput cost.

The general shape of the decision: the more your storage stack lies about durability — consumer SSDs with volatile write caches, RAID controllers without a BBU, virtualised storage with writeback caching — the further up this scale you should be. Enterprise SSDs with power-loss protection are what the default assumes.

What happens if you put the datastore on an NFS or CIFS share?

It works, and it is the configuration that produces the most PBS support threads. Three things go wrong, in order of how long they take to notice.

Immediately: throughput is fine and metadata is not. Backup itself is mostly sequential chunk writes and will look acceptable. GC, verify and prune are millions of small metadata operations over a network filesystem with a round trip on each one. A GC that takes twenty minutes on local SSD can take a day.

Within weeks: the atime problem. Many NFS exports are mounted noatime or served by an appliance that treats atime as advisory. See the callout above — this is the failure mode where garbage collection silently stops reclaiming, and you find it when the datastore fills.

At the worst moment: locking and consistency. PBS assumes POSIX semantics for the chunk store. Network filesystems provide a weaker version of that, and the divergence shows up under concurrency — several backup jobs plus a GC plus a verify — which is exactly the load profile of a busy backup window.

If shared storage is the only option, prefer a block-level export (iSCSI, FC) with a local filesystem on top over a file-level share. That gives PBS the POSIX semantics it expects. And in either case, budget the verify job time before committing.

The S3 backend, and what it does not remove

PBS 4 can place a datastore on S3-compatible object storage. The datastore is created against a configured endpoint and a local cache path:

Configuration changean S3-backed datastore still needs local disk— The cache path is not optional and is not small. Plan 64-128 GiB of local persistent storage for it.

# The endpoint is configured first, under Configuration > Remotes > S3
# Endpoints in the GUI, or via the API.
proxmox-backup-manager datastore create archive /var/cache/pbs-archive \
--backend type=s3,client=s3-endpoint-1,bucket=backups-example

This is genuinely useful for a long-retention tier. It is not a way to avoid buying disks: the local cache is required, object-store latency makes it a poor primary tier for restore-time-sensitive workloads, and egress pricing turns a full-cluster restore into a line item somebody has to approve during an outage.