Multiple clusters on one host · PostgreSQL

The first lesson of this course established that databases inside one cluster are tenants rather than isolated services: they share a restart window, a connection budget, a write-ahead log and a restore point. When those shared properties are the problem, the answer is a second cluster.

A second cluster on the same host is genuinely a second server. It has its own postmaster, its own data directory, its own port, its own configuration, its own WAL and its own version. Two clusters on one machine share nothing that PostgreSQL manages.

Creating one

On Debian and Ubuntu the packaging makes this a single command, because the layout was designed for it.

Configuration changea second cluster alongside an existing one— Port 5433 was allocated automatically because 5432 was in use. Captured from a live run.

$ pg_createcluster 18 reporting

selecting default "shared_buffers" ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
Ver Cluster   Port Status Owner    Data directory                   Log file
18  reporting 5433 down   postgres /var/lib/postgresql/18/reporting /var/log/postgresql/postgresql-18-reporting.log

Elsewhere, the same result is initdb into a new directory plus a service unit and a port assignment done by hand:

# The manual equivalent, on any layout.
NEWDATA=/var/lib/pgsql/18/reporting
initdb --pgdata="$NEWDATA" --encoding=UTF8 --data-checksums
# Then set port = 5433 in that cluster's postgresql.conf,
# and create a service unit that passes -D "$NEWDATA".

The work the wrapper saves is not the initdb; it is remembering to allocate a free port, to give the cluster a distinct log file, and to create a unit that will start it at boot.

Addressing the right one

Every client tool needs to be told which cluster it means, and the mechanism is the port or the socket.

# By port, over TCP
psql -h 127.0.0.1 -p 5433 -U postgres -c 'SHOW data_directory'

# By port, over the Unix socket directory
psql -h /var/run/postgresql -p 5433 -U postgres -c 'SHOW data_directory'

# By environment, which is how scripts should do it
export PGPORT=5433
psql -U postgres -c 'SHOW data_directory'

Read-only / Safetwo clusters answering for themselves on one host— Same host, same binaries, two servers. Captured from a live run.

$ psql -p 5433 -tAc 'SHOW data_directory'; psql -p 5432 -tAc 'SHOW data_directory'

/var/lib/postgresql/18/reporting
/var/lib/postgresql/18/docker

SHOW data_directory is the confirmation step that belongs at the top of any procedure run on a multi-cluster host. The port you passed is what you asked for; the data directory is what you got, and on a host where a colleague has changed a port assignment those are not always the same.

On a multi-cluster host, every destructive command needs a target check first

The commands that end a cluster — pg_dropcluster, DROP DATABASE, a restore that overwrites a data directory — take a target as an argument, and on a single-cluster host getting it wrong is nearly impossible. On a multi-cluster host it is one word.

Make the confirmation explicit rather than assumed:

psql -p "$PGPORT" -tAc 'SHOW data_directory'
psql -p "$PGPORT" -tAc 'SELECT current_database(), inet_server_port()'

inet_server_port() is useful because it reports the port the server believes it is on, which catches the case where an environment variable and a connection service file disagree and you have connected somewhere other than you intended.

What is isolated, and what is not

This is the table that decides whether a second cluster solves your problem.

Isolated by a separate cluster	Still shared
Restart and reload windows	CPU
`max_connections` budget	Physical memory
Configuration, including `shared_buffers`	The OS page cache
The write-ahead log	Disk bandwidth and IOPS
Backup and restore point	The filesystem, and its free space
Major version	Kernel limits: file descriptors, shared memory, semaphores
Roles and authentication	The network interface
Crash blast radius	The host itself

The left column is why you would do it. The right column is why it is not a substitute for a second machine.

A reporting cluster that runs an enormous analytical query will not consume the transactional cluster’s connection slots or force it to restart, and that is a real and valuable separation. It will absolutely consume the disk bandwidth, the page cache and the CPU that the transactional cluster was relying on.

Splitting into two clusters to solve a resource contention problem

The reasoning sounds right — the reporting workload is hurting the transactional workload, so separate them — and it addresses the wrong axis.

Two clusters on one host contend for exactly the same physical resources as two databases in one cluster did. The heavy query still reads the same disks, still evicts the same page cache, still occupies the same CPUs. What changes is that it now does so through a second buffer pool, which means the same data can be cached twice and the total memory available for caching is lower than it was.

Separate clusters solve scheduling and failure-domain problems: independent restarts, independent restore points, independent versions, independent connection budgets, and a crash in one not taking the other down. They do not solve contention.

Contention is solved by separate hosts, by cgroup limits that bound what each cluster may consume, or by fixing the query. Part XI covers the cgroup approach and Part X covers the query.

The failure modes specific to multi-cluster hosts

Port collisions and reassignments. A cluster whose port is changed without updating the clients that reach it fails in a way that looks like an outage. Worse, if two clusters swap ports, connections succeed and reach the wrong database. inet_server_port() and SHOW data_directory are the defence.

Kernel limits are shared and additive. Each cluster wants file descriptors, semaphores and shared memory. A host sized for one cluster can refuse to start the second, and the error names the kernel resource rather than PostgreSQL. Part XI covers the specific limits.

Memory over-commitment is easy and silent. Two clusters each configured with shared_buffers at a quarter of host memory are using half of it before a single query runs, and each will additionally allocate work_mem per operation per backend. The OOM killer resolves this eventually and badly.

Backups multiply. Each cluster needs its own backup configuration, its own WAL archive destination and its own restore test. A second cluster that nobody added to the backup schedule is the most common serious consequence, and it is invisible until a restore is needed.

Log files diverge. Each cluster writes its own log. A log aggregation configuration that names one file silently stops covering the estate the moment a second cluster exists.

Why two postmasters cannot share a data directory, and how that is enforced

The isolation between clusters is not a convention; it is enforced, and the enforcement is worth understanding because it is what makes postmaster.pid more than a convenience file.

When the postmaster starts, it creates postmaster.pid in the data directory containing its own PID and the shared memory key it is using. If the file already exists, it reads it and asks two questions: is a process with that PID still alive, and is the shared memory segment named in the file still attached by anything?

If both answers are no, the file is stale and is removed automatically — which is why a genuinely crashed cluster restarts without intervention. If either answer is yes, the postmaster refuses to start, because the alternative is two servers writing to the same files with no coordination.

That second check, on the shared memory segment, is the one that matters on a multi-cluster host. A PID can be reused by an unrelated process after a reboot, so PID alone is not sufficient evidence. The shared memory attachment count is much stronger evidence that backends from a previous instance are still alive.

The consequence for operators is the one from the data-directory lesson, and it is worth repeating in this context: deleting postmaster.pid to clear a startup refusal bypasses both checks. On a host running several clusters, where directories and ports are easy to confuse, the chance that the refusal was protecting you from something real is materially higher.

Choosing between the three options

Requirement	Separate databases	Separate clusters	Separate hosts
Logical separation of data	✓	✓	✓
Independent connection budget	✗	✓	✓
Independent restart window	✗	✓	✓
Independent restore point	✗	✓	✓
Different major versions	✗	✓	✓
Independent CPU, memory, disk	✗	✗	✓
Survives host loss	✗	✗	✓
Cross-database transactions	✗	✗	✗

Note the last row. Transactional consistency across the boundary is lost as soon as you leave a single database, and it does not come back at any of the higher tiers. That is the cost to weigh against everything in the left column, and it is the reason not to split an application’s own data.

The genuinely common good use for a second cluster is a major version upgrade: pg_upgrade requires both versions present, and for the duration of the upgrade you have two clusters on one host by construction. Part XVII covers that case in full.

Production discipline

Confirm the target before every destructive command. SHOW data_directory and inet_server_port() cost nothing and are the only reliable evidence of which cluster you reached.
Add every new cluster to backup, monitoring and log aggregation on the day it is created. A cluster nobody backs up is the failure that surfaces at restore time.
Do not use separate clusters to solve resource contention. They isolate scheduling and failure, not CPU, memory or disk, and two buffer pools cache the same data twice.
Sum shared_buffers across all clusters before sizing the host, and remember work_mem is allocated per operation per backend on top of that.
Set PGPORT explicitly in scripts rather than relying on the default, so that a script cannot silently address the wrong cluster.

Cross-course references

Linux for Production Sysadmins — Part XXXVII (Resources) covers the kernel limits that two clusters share and Part VII (systemd) covers writing the unit a manually created cluster needs.
Docker & Containers — Part XV (Resource controls) covers bounding what a workload may consume, which is the actual remedy for the contention that separate clusters do not solve.
Proxmox — Part IX (VMs) covers the case where separate hosts is the right answer and the hosts are virtual.