RunBook AcademyPostgreSQL

PostgreSQL · Operational reference

Toolkit

Operational artefacts for PostgreSQL: checklists for cadences, runbooks for incidents, hands-on labs for skills, and break/fix scenarios for troubleshooting reflexes. None of this replaces reading the corresponding lessons — these are the artefacts you keep open in a second tab during real work.

Runbooks: 26
Checklists: 15
Labs: 26
Break/Fix: 30
Assessments: 2

Operational checklists

Repetitive tasks performed on a schedule.

Before deployment

Monthly

Quarterly

Annual

PostgreSQL Disaster Recovery Readiness Review21 items→

As needed

Runbooks

Step-by-step procedures. Treat the rollback as part of the procedure — never skip it.

Critical risk

High risk

Medium risk

Low risk

Hands-on labs

Time-boxed exercises.

C · Simulation

Break/Fix scenarios

Troubleshooting drills. Each scenario gives symptoms and evidence, then hides the solution behind a reveal.

pg-startup

intermediateThe database did not come back after the kernel patch, and the change that broke it was six weeks old~35 min · 7 symptoms→

pg-hba

intermediateThe reporting service could not connect, and the rule granting it access was correct and useless~35 min · 7 symptoms→

pg-idle-in-transaction

advancedEvery table in the cluster started bloating, and autovacuum was working perfectly~40 min · 7 symptoms→

pg-lock

advancedA schema migration made the whole application stop, and the table it locked was not busy~40 min · 7 symptoms→

pg-archive

advancedThe disk filled on a database that had not grown, and the backup job reported success every night~40 min · 7 symptoms→

pg-replication-slot

advancedA replica was decommissioned in March and took the primary down in June~40 min · 7 symptoms→

pg-backup

intermediateThe restore finished, the row counts matched, and the application could not read a single table~35 min · 7 symptoms→

pg-deadlock

advancedDeadlocks went from four a day to nine hundred, and nothing in the code had changed~35 min · 7 symptoms→

pg-statistics

advancedA dashboard query went from 40 milliseconds to eleven minutes, and nothing was deployed~40 min · 7 symptoms→

pg-replica-down

intermediateThe failover target had not been replicating for six weeks, and the dashboard was green the whole time~35 min · 7 symptoms→

pg-disk-full

intermediateThe cluster PANICked at 02:14 with no space left on device, and the first suggestion in the channel was to delete WAL files~40 min · 7 symptoms→

pg-wraparound

advancedThe warning said the database must be vacuumed within eleven million transactions, and every VACUUM anybody ran changed nothing~45 min · 7 symptoms→

pg-auth

intermediateThe password was rotated three times and authentication kept failing, and every test from the database host succeeded~40 min · 7 symptoms→

pg-tls

intermediateA packet capture found database credentials in cleartext on a cluster where TLS had been enabled and verified eight months earlier~40 min · 7 symptoms→

pg-connections

intermediateNobody could connect, including the on-call engineer, and max_connections had already been raised twice that year~40 min · 7 symptoms→

pg-connection-storm

advancedA four-second storage stall turned into a two-hour outage, and the database never recovered on its own after the storage came back~45 min · 7 symptoms→

pg-long-transaction

advancedAutovacuum ran hundreds of times and removed nothing, on tables the nightly report has never touched~40 min · 7 symptoms→

pg-autovacuum

advancedAutovacuum had been starting on the same table every night for a year and had never once finished~40 min · 7 symptoms→

pg-bloat

intermediateA VACUUM FULL was run to reclaim disk space during business hours and every query on the table stopped for eleven minutes~40 min · 7 symptoms→

pg-index

intermediateWrite latency rose 30 percent after an index build that everyone believed had been rolled back~35 min · 7 symptoms→

pg-temp-files

intermediateRaising work_mem to stop queries spilling to disk made the reports slower and nearly filled the volume~40 min · 7 symptoms→

pg-checkpoint

intermediateWrite latency spiked every ninety seconds and the log had been saying why since the batch job was introduced~40 min · 7 symptoms→

pg-replication-lag

intermediateThe replication lag alert fired every Sunday for a year, and the week it was telling the truth it had already been muted~35 min · 7 symptoms→

pg-sync-replication

advancedA standby was rebooted for a kernel patch and every write on the primary stopped, on a cluster built for high availability~45 min · 7 symptoms→

pg-oom

advancedOne analyst query was killed by the kernel and every other session on the cluster died with it~40 min · 7 symptoms→

pg-promotion

intermediateThe failover completed in under a second and the team spent nineteen minutes deciding whether it had worked~35 min · 7 symptoms→

pg-split-brain

advancedTwo databases accepted writes for fifty minutes, and resolving it discarded a hundred and twenty committed orders~50 min · 7 symptoms→

pg-pitr

advancedA point-in-time recovery restored the deleted rows and then replayed the deletion, and the second attempt had to start from the base backup again~45 min · 7 symptoms→

pg-corruption

advancedOne query started failing with invalid page in block 5, and somebody found a setting that made the error go away~50 min · 7 symptoms→

pg-pooler

advancedAfter the pooler was switched to transaction mode, a scheduled job started blocking unrelated clients on a lock nobody was holding~45 min · 7 symptoms→

Assessments

Production-readiness examinations. Each combines auto-scored questions with scenario-based rubrics you can use to grade yourself.