Automated installation with answer files · Proxmox VE

The interactive installer produces a node. It does not produce a procedure. Every choice made in front of it — the ZFS layout, the ashift, the swap size, the management VLAN — exists only in the memory of whoever clicked through it, and the next node is built from that memory.

Two situations turn that into a real problem. Building a fleet, where nodes must be identical and are not. And rebuilding a failed node at 03:00, where the person at the keyboard is not the person who built it and every answer they guess is a divergence they will discover weeks later.

The answer file is the fix. It is a TOML document describing every choice the installer would ask about, and the installer applies it without interaction.

The tool

Configuration changeinstall the assistant— Installs on any Debian or Proxmox machine, not on the target. xorriso is required for prepare-iso and is not pulled in automatically.

apt update
apt install proxmox-auto-install-assistant xorriso

proxmox-auto-install-assistant has six subcommands, and you will use all of them:

Subcommand	What it does
`validate-answer`	Checks an answer file for syntax and schema errors
`device-info`	Prints UDEV properties for disks or network interfaces
`device-match`	Tests a filter expression against the hardware present
`prepare-iso`	Produces a bootable ISO or PXE tree with the fetch mode baked in
`inspect-iso`	Reports what a prepared ISO was configured to do
`system-info`	Prints the machine identification an HTTP answer server receives

The answer file

Four sections. [global], [network] and [disk-setup] are required; the rest are optional.

[global]
keyboard = "en-us"
country = "gb"
fqdn = "pve01.example.com"
mailto = "platform-alerts@example.com"
timezone = "Europe/London"
root-password-hashed = "REPLACE_ME-with-mkpasswd-output"
root-ssh-keys = [
  "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAREPLACEME platform-team",
]
reboot-on-error = false

[network]
source = "from-dhcp"

[disk-setup]
filesystem = "zfs"
disk-list = ["sda", "sdb"]

[disk-setup.zfs]
raid = "raid1"
ashift = 12
compress = "lz4"
arc-max = 16384

That is a complete, working answer file. Everything below is about making it correct for a fleet rather than for one machine.

`[global]`

keyboard, country, timezone and mailto are what the interactive installer asks. fqdn needs more thought than it looks.

root-password versus root-password-hashed. Set one, never both. The plaintext form puts a working root password in a file that will be copied into a repository, a PXE tree, and probably a ticket. Generate a hash instead:

Read-only / Safegenerate a password hash for the answer file— Produces a yescrypt hash on Debian 13. The hash is what goes in the answer file; the plaintext never leaves your terminal.

mkpasswd --method=yescrypt

A hash in an answer file is still a credential — it is offline-crackable — so it belongs in the same store as any other secret and not in a public repository. What it is not is a password that works if someone reads the file.

root-ssh-keys is what makes the node usable without the password at all. Populate it and the root password becomes a console-only fallback.

reboot-on-error = false is the setting to keep at its default while you are developing the answer file. On a failure the installer stays on screen with the error visible; with it set to true, a failing node reboots into the installer again and loops, showing you nothing.

A per-node FQDN is the one field that stops an answer file being reusable — which is what fqdn.source exists for:

[global]
fqdn.source = "from-dhcp"
fqdn.domain = "example.com"

The installer takes the hostname from DHCP option 12 and the domain from option 15, falling back to fqdn.domain when the DHCP server does not supply one. Now the same answer file builds every node in the rack, and identity comes from the DHCP reservation, which is already where your IPAM lives.

subscription-key is available and implies the proxmox-first-boot package. Putting a subscription key in a file that gets copied around is a decision to make deliberately.

`[network]`

Two modes. source = "from-dhcp" takes everything from DHCP. source = "from-answer" requires cidr, gateway and dns, and a filter to pick the interface.

[network]
source = "from-answer"
cidr = "192.0.2.11/24"
gateway = "192.0.2.1"
dns = "192.0.2.53"
filter.ID_NET_NAME_MAC = "*e43d1afa379a"

DHCP with a reservation is generally the better answer for a fleet: the address still comes from a controlled place, and the answer file stays identical across nodes.

Interface name pinning, available since PVE 9.1, is the section that prevents a class of incident that predates automation entirely:

[network.interface-name-pinning]
enabled = true

[network.interface-name-pinning.mapping]
"24:8a:07:1e:05:bc" = "lan0"
"24:8a:07:1e:05:bd" = "lan1"
"b4:2e:99:ac:ad:b4" = "mgmt"

Without pinning, the installer names interfaces nic0, nic1, nic2. With it, a MAC gets a stable name you chose. That matters because /etc/network/interfaces refers to interfaces by name, and a name that moves between physical ports after a firmware update, a card swap, or a BIOS change produces a node whose management traffic is on the wrong wire. Pinning ties the name to the MAC, so the mapping survives.

`[disk-setup]`

Choose the filesystem, then the disks, then the per-filesystem options.

[disk-setup]
filesystem = "zfs"
filter.ID_MODEL = "SAMSUNG_MZ*"
filter.ID_SERIAL = "*"
filter-match = "all"

[disk-setup.zfs]
raid = "raid10"
ashift = 12
compress = "zstd"
checksum = "sha256"
arc-max = 16384
hdsize = 400

filesystem takes ext4, xfs, zfs or btrfs — the last being a technology preview and not a production choice.

Disks are selected by disk-list or by filter, never both. filter-match takes any (the default) or all, deciding whether the filters combine with OR or AND.

For ZFS, raid accepts raid0, raid1, raid10, raidz-1, raidz-2 and raidz-3. ashift = 12 is 4K sectors and is correct for essentially every current drive. arc-max deserves a value on every hypervisor, in MiB: unbounded ARC competes with guest memory, and the answer file is the earliest and cheapest place to bound it.

For ext4 and xfs the corresponding section is [disk-setup.lvm] with hdsize, swapsize, maxroot, maxvz and minfree — the same knobs the interactive installer exposes under Advanced Options.

Filters that survive a hardware change

A disk-list of ["sda", "sdb"] is a bet that kernel enumeration order is stable. It is not: add an HBA, change a backplane, or upgrade the kernel, and sda can become the data disk you did not want to overwrite.

Filters match UDEV properties instead. Find out what your hardware actually reports:

Read-only / Safewhat does this hardware report?— Read-only. Run from a live PVE installer in debug mode, or from an identical already-built node. Prints the UDEV properties available for filtering.

proxmox-auto-install-assistant device-info -t disk
proxmox-auto-install-assistant device-info -t network

Useful disk properties are DEVNAME, ID_SERIAL_SHORT, ID_WWN, ID_MODEL and ID_SERIAL. For network interfaces: ID_NET_NAME, ID_NET_NAME_MAC, ID_VENDOR_FROM_DATABASE and ID_MODEL_FROM_DATABASE.

Filters support glob wildcards: ? for one character, * for any run, [abc] and [0-9] for classes, [!a] for negation.

Then test the expression before you trust it:

Read-only / Safedoes the filter select what you think?— Read-only. Evaluates a filter against the hardware present and prints the matching devices. The only way to find out that a filter matches three disks instead of two without discovering it during an install.

proxmox-auto-install-assistant device-match disk ID_MODEL='SAMSUNG_MZ*'

Getting the answer file to the installer

Three fetch modes, baked into the ISO at preparation time.

`--fetch-from iso`

The answer file is embedded in the ISO. Simplest, and correct when the answer file is genuinely per-node or per-model.

Configuration changeembed the answer file in the ISO— Writes a new ISO containing the answer file. Validate first: prepare-iso will happily embed a file that the installer later rejects.

proxmox-auto-install-assistant validate-answer answer.toml

proxmox-auto-install-assistant prepare-iso \
proxmox-ve_9.2-1.iso \
--fetch-from iso \
--answer-file answer.toml \
--output pve-auto-rack1.iso

The cost is that the credentials are inside the ISO. Treat a prepared ISO as a secret-bearing artefact — inspect-iso will show what it carries, and --show-sensitive will show the values.

`--fetch-from partition`

The ISO is generic; the answer file lives on a separate FAT partition labelled PROXMOX-AIS. One ISO, many answer files, no server.

Configuration changeprepare a generic ISO that looks for a labelled partition— The resulting ISO contains no answer file. It searches attached block devices for a partition labelled PROXMOX-AIS and reads answer.toml from it.

proxmox-auto-install-assistant prepare-iso \
proxmox-ve_9.2-1.iso \
--fetch-from partition \
--output pve-auto-generic.iso

Destructiveprepare the answer USB stick— Formats a partition on the target device. Verify the device path with lsblk first: this writes a filesystem over whatever is there, and getting the letter wrong destroys the wrong disk.

lsblk -o NAME,SIZE,MODEL,SERIAL

USB_PART=/dev/sdz1

mkfs.vfat "$USB_PART"
fatlabel "$USB_PART" 'PROXMOX-AIS'
mkdir -p /mnt/ais
mount "$USB_PART" /mnt/ais
cp answer.toml /mnt/ais/answer.toml
sync
umount /mnt/ais

The file must be named exactly answer.toml. A custom partition label is possible with --partition-label since PVE 8.3-1.

`--fetch-from http`

The installer asks a server for its answer file, and the server decides what to send based on who is asking. This is the fleet mode.

Configuration changeprepare an ISO that fetches over HTTPS— Bakes the answer-server URL and the expected certificate fingerprint into the ISO. The fingerprint is what stops the installer trusting any server that answers.

proxmox-auto-install-assistant prepare-iso \
proxmox-ve_9.2-1.iso \
--fetch-from http \
--url 'https://provisioning.example.com/answer' \
--cert-fingerprint '04:42:97:27:F6:29:2F:9F:3D:7F:13:11:C8:E2:F5:5F:84:03:95:D9:F5:14:72:7C:9E:90:47:03:D2:96:2B:EC' \
--answer-auth-token 'REPLACE_ME-provisioning-token' \
--output pve-auto-http.iso

The URL and fingerprint do not have to come from the ISO. The installer will also take them from DHCP option 250 (URL) and option 251 (fingerprint), or from DNS TXT records at proxmox-auto-installer.{search-domain} and proxmox-auto-installer-cert-fingerprint.{search-domain}. That makes one truly generic ISO possible, with the environment supplying the rest.

The installer POSTs machine identification to the server — the same data system-info prints — so the server can return an answer file matched to this specific machine. Run system-info on a node you already have to see the shape of what your server will receive.

For PXE, --pxe and --pxe-loader ipxe write a boot tree to --output instead of an ISO.

First-boot hooks

An installed node is not a configured node. [first-boot] runs an executable the first time the system comes up.

[first-boot]
source = "from-iso"
ordering = "fully-up"

Configuration changeembed a first-boot script— Adds an executable that runs once on first boot. Maximum size is 1 MiB; scripts need a shebang and binaries must be statically linked x86_64.

proxmox-auto-install-assistant prepare-iso \
proxmox-ve_9.2-1.iso \
--fetch-from iso \
--answer-file answer.toml \
--on-first-boot first-boot.sh \
--output pve-auto-rack1.iso

ordering decides when it runs: before-network, network-online, or fully-up (the default). Anything that needs to reach a repository or a configuration-management server needs at least network-online.

Keep the hook small. Its job is to make the node reachable by your real configuration management — install the agent, register with it, join it to the right group — not to be your configuration management. A five-hundred-line first-boot script is a second, undebuggable provisioning system that only runs once and logs nowhere convenient.

Under the hood

What the automated installer is, and what it is not.

It is the same installer. The low-level installation logic that partitions disks, creates the pool, unpacks the base system and installs the bootloader is shared with the graphical and terminal installers. The automated path replaces the interaction layer: instead of collecting answers from a person, it reads them from TOML and validates them against the same schema the UI enforces.

That shared-logic property is why filesystem = "zfs" produces exactly the layout the ZFS-root lesson describes, and why the bootloader choice is made the same way — systemd-boot on ZFS root without Secure Boot, GRUB otherwise. The answer file does not select a bootloader and there is no key for one, because the installer derives it.

Three consequences of the schema being validated up front.

validate-answer catches structure, not intent. It confirms the TOML parses, the keys exist, and the values are of the right type and within the allowed set. It cannot know that raid = "raid1" with a filter matching four disks is not what you meant, or that your cidr collides with another node. Filter behaviour is device-match, and everything else is a review.

An answer file is version-coupled to the ISO. Keys have been added over time — [first-boot] and the post-installation webhook in PVE 8.3-1, interface name pinning in 9.1-1. An answer file written for a newer installer and used with an older ISO fails on the unknown key. Validate with the assistant version matching the ISO you are preparing, which in practice means keeping the assistant current.

The installer runs before any of your tooling exists. There is no configuration management, no monitoring, no log shipping. A failure at this stage is visible only on the console, which is why reboot-on-error = false matters and why out-of-band console access is a prerequisite for automated installation rather than a nicety. Without IPMI or equivalent, an automated install that fails is a trip to the rack regardless of how good the automation is.

The post-installation webhook closes part of that gap. Configured under [post-installation-webhook] with a url, an optional cert-fingerprint and an optional auth-token, it POSTs a JSON document containing the disks, network interfaces, machine-id and SSH host keys of the freshly installed system. That is enough to record the node in an inventory and to pin its SSH host keys before anyone connects — which turns the first SSH connection from trust-on-first-use into a verified one.

What happens if the answer file is subtly wrong for one node in a rack?

The interesting failures are the ones that install successfully.

A wrong disk filter that matches too few disks produces a node with a smaller pool than its neighbours. Nothing fails. Nothing alerts. The node joins the cluster, takes its share of guests, and fills up months later while identical-looking nodes have room — and the capacity model, which assumed uniformity, has been wrong the whole time.

A wrong arc-max produces a node whose ARC competes with guest memory. Its guests are slower under load than the same guests on other nodes. This surfaces as an application complaint about specific VMs, gets diagnosed as a workload problem, and moves around the cluster as guests migrate.

A wrong network filter picks the wrong NIC, and the node comes up on the right subnet through the wrong physical path — a 1G port instead of a 25G one, or a port that is not in the bonded pair. Cluster traffic works. Migration is slow. Corosync is on a link with different characteristics from every other node, which is exactly the condition that produces intermittent quorum instability months later.

What these have in common is that they are all divergence, and divergence is the failure automation is supposed to eliminate. The answer file made the build reproducible without making it verified.

The fix is a post-install verification step, and it belongs in the same repository as the answer file. After a node is built, assert the facts the answer file was supposed to produce: pool topology from zpool status, ARC limit from /sys/module/zfs/parameters/zfs_arc_max, the active interface and its speed, the FQDN and the timezone. Compare against the intended values rather than against nothing.

The stronger version is to compare a new node against an existing one rather than against a written spec, because the written spec drifts too. Either way, the check has to exist: an unverified automated build is a faster way to produce inconsistency, not a way to avoid it.

Common mistakes

disk-list with kernel names. sda is not stable across an HBA change or a kernel upgrade. Filter on ID_WWN or ID_SERIAL.
A filter never tested with device-match. The installer consumes every disk it selects, without a confirmation step.
root-password in plaintext. Use root-password-hashed, and treat even that as a secret.
reboot-on-error = true while developing. The node loops and shows you nothing.
A prepared ISO stored like an ordinary artefact. It contains credentials; inspect-iso --show-sensitive proves it.
HTTP mode without --cert-fingerprint. The installer will take its entire configuration, including the root credential, from whatever answers.
A first-boot script that is the provisioning system. It runs once, logs awkwardly, and cannot be re-run. Bootstrap into real configuration management instead.
No post-install verification. Reproducible is not the same as correct.

Key takeaways

The answer file is the procedure. Without it the build lives in somebody’s memory and the rebuild diverges.
[global], [network], [disk-setup] are required; fqdn.source = "from-dhcp" is what makes one file build a whole rack.
Select disks and NICs by UDEV property, and prove the filter with device-match before it touches hardware.
Three fetch modes: iso for per-node, partition for one generic ISO plus a labelled stick, http for a fleet with a server that answers.
HTTP mode needs the certificate fingerprint, from the ISO, DHCP options 250/251, or DNS TXT records.
Interface name pinning ties a name to a MAC, so management traffic stays on the wire you chose.
First-boot hooks bootstrap into configuration management; they are not configuration management.
Verify after the build. Automation makes divergence faster, not impossible.

Knowledge check

Why this matters

The argument for automated installation is usually made in terms of speed, and speed is the least valuable thing it delivers. Installing Proxmox interactively takes about ten minutes and you do it rarely.

What it delivers is that the build becomes a reviewable artefact. The ashift, the ARC limit, the pool topology, the management interface and the timezone stop being decisions someone made once and become lines in a file that lives in version control, has a commit history, and can be diffed between what you meant to build in March and what you are building now.

That changes three things. A rebuild at 03:00 produces the same node as the original, by construction, rather than by the recollection of whoever is awake. A hardware refresh that introduces a new disk model becomes a filter change with a review rather than a fresh set of decisions made under deadline. And when a node behaves differently from its neighbours, “how was this one built” has an answer.

The caveat is the one the failure section makes: an answer file guarantees repeatability, not correctness. A wrong ARC limit reproduces perfectly across twelve nodes. Which is why the answer file and the post-install verification belong in the same repository, reviewed together — the file states the intent, and the check proves the machine matched it.