How to use this checklist
Before the cluster carries data, and again whenever its storage changes.
Several items here are cheap now and expensive later: the WAL directory’s location is fixed until the cluster can be stopped, and checksums cost a scan of the whole cluster to add afterwards. Read those first.
max_wal_size is not a cap
Headroom in hours, not percent
The alert threshold that matters is derived from the measured WAL generation rate:
free bytes / bytes of WAL per hour = hours of headroomIn one incident, 95 percent of a 20 GB volume was sixteen minutes. The alert fired, somebody acknowledged it, and the cluster stopped before anybody could act.
A percentage on a volume of unknown size and unknown fill rate is not a threshold.
Checksums, and what their absence looks like
Two things follow for verification:
- Checksums are verified on read from disk, not from shared buffers. A check against a warm running cluster can pass on a damaged page that happens to be cached.
- Run
pg_checksums --checkagainst a restored backup. It verifies the backup and the checksums in one pass and costs production nothing.
The two prohibitions, in writing, before the incident
Two numbers worth measuring before go-live
Storage latency under a representative write load. Without a baseline, “the disks feel slow” during an incident is an opinion.
The knee of the throughput curve. Measured on one four-core cluster: peak at 64 concurrent clients, then 28 percent less throughput and 8.6 times the latency by 400. That number sizes the connection pool, and having it turns pool sizing from an argument into arithmetic.
Where the numbers come from
Free space comes from df per filesystem, not from a single figure for
the host. Database sizes come from pg_database_size summed across
databases; table and index sizes from pg_total_relation_size. Growth
comes from retained series, because a single reading gives a level and
the item asks about a trend.
Whether the device honours a flush is established from the storage documentation and, where possible, from a measurement — not assumed from the fact that PostgreSQL issues the flush.
Access this needs
Read access to every filesystem the cluster uses, separately: the data
directory, pg_wal, the log destination, any tablespace, and the local
staging path a backup or archive command writes to. “The disk” is not a
unit of measurement here and the items depend on telling them apart.
A role holding pg_monitor for the database sizes and the archiver
counters, and read access to the storage layer’s own view — the volume,
the LVM configuration, or the distributed-storage cluster’s health.
Read access to the mount options, because whether a flush is honoured is a property of the mount and the device rather than of PostgreSQL.
What the review produces
A dated record naming the reviewer, every filesystem the cluster touches
with its size, usage and growth rate, and the disposition of every item.
Attach the two prohibitions in writing — that nobody deletes from
pg_wal, and that nobody disables fsync — with the name of whoever
acknowledged them, because those are the two decisions taken under
pressure that cannot be undone.
A pg_wal filesystem shared with the data directory is a finding even
when there is plenty of room, and goes to the service owner.
Sign-off
- Reviewer: ________________ Date: ___________
- Database owner: ___________ Date: ___________
- Service owner: ____________ Date: ___________
Every critical item must pass. A failing critical item is a blocker, not a note for the next sprint: record the date, the reviewer, the disposition of every item that did not pass, and the name of whoever accepted the residual risk.