Skip to main content
RunBook Academy

PostgreSQLXIV · Replication, Slots and Read ReplicasReplication

WAL sender and WAL receiver

Intermediate⏱ ~25 minpsql

What you'll learn

  • Read pg_stat_replication and pg_stat_wal_receiver correctly
  • Configure the replication connection and its authentication
  • Diagnose a standby that will not connect
  • Understand what the replication protocol carries in each direction

Prerequisites

Verified against PostgreSQL 18.x · PostgreSQL (comparison targets) 17.11, 16.15 · PostgreSQL (support calendar) 18, 17, 16, 15, 14 supported · pgBackRest 2.59.1 · PgBouncer 1.25.2 · Patroni 4.1.5 · Ubuntu (host baseline) 26.04 LTS · 2026-08-27

Not yet marked complete on this device.

The walsender and walreceiver are the two ends of one TCP connection. Each has a view describing it, and the views are not symmetrical.

The primary’s view

Read-only / Safepg_stat_replication, every column
$ SELECT * FROM pg_stat_replication;
-[ RECORD 1 ]----+------------------------------
pid              | 124
usesysid         | 16384
usename          | repl
application_name | walreceiver
client_addr      | 172.26.0.3
client_port      | 46864
backend_start    | 2026-08-27 21:57:00.185689+00
backend_xmin     |
state            | streaming
sent_lsn         | 0/5000060
write_lsn        | 0/5000060
flush_lsn        | 0/5000060
replay_lsn       | 0/5000060
write_lag        | 00:00:00.010948
flush_lag        | 00:00:00.011917
replay_lag       | 00:00:00.011954
sync_priority    | 0
sync_state       | async
reply_time       | 2026-08-27 21:57:04.824462+00

One row per connected standby. No row means no standby is connected, which is the single most important thing this view tells you and the easiest to miss, because an empty result set does not look like an alert.

-- the check that matters, written so that zero standbys is a failure
SELECT count(*) AS connected_standbys FROM pg_stat_replication;

state moves through startup, catchup and streaming. A standby stuck in catchup is reading through a backlog; one in streaming is current. Lesson XIV-04 covers the four LSNs and three lag columns.

The standby’s view

Read-only / Safepg_stat_wal_receiver, every column
$ SELECT * FROM pg_stat_wal_receiver;
-[ RECORD 1 ]---------+---------------------------------------------
pid                   | 32
status                | streaming
receive_start_lsn     | 0/5000000
receive_start_tli     | 1
written_lsn           | 0/5033010
flushed_lsn           | 0/5033010
received_tli          | 1
last_msg_send_time    | 2026-08-27 21:57:20.223009+00
last_msg_receipt_time | 2026-08-27 21:57:20.22313+00
latest_end_lsn        | 0/5033010
latest_end_time       | 2026-08-27 21:57:20.223009+00
slot_name             | standby1
sender_host           | rbpg-prim
sender_port           | 5432
conninfo              | user=repl password=******** dbname=replication
                      host=rbpg-prim port=5432
                      fallback_application_name=walreceiver ...

last_msg_send_time and last_msg_receipt_time are the pair to watch for a stalled connection: a large and growing gap between the last message the primary sent and now means the connection is not delivering, even when status still says streaming.

Configuring the connection

Three things must line up.

On the primary, a role with the REPLICATION attribute:

CREATE ROLE repl WITH REPLICATION LOGIN PASSWORD '...';

REPLICATION is a distinct privilege from superuser and from any table grant. A replication role needs no SELECT on anything, because it never reads through the SQL layer — which makes it a good example of least privilege that is genuinely achievable.

In pg_hba.conf, a line for the special database name replication:

# TYPE  DATABASE      USER  ADDRESS       METHOD
host    replication   repl  10.0.0.0/24   scram-sha-256

replication here is not a database. It is a keyword meaning “replication connections”, and a host all all line does not cover it. This is the most common reason a standby cannot connect, and the error message says so plainly.

On the standby, primary_conninfo, normally written by pg_basebackup -R.

When a standby will not connect

The failures are distinguishable from their messages, and each names its own fix.

MessageCause
no pg_hba.conf entry for replication connectionMissing replication line
password authentication failed for user "repl"Wrong password, or .pgpass not read
must be superuser or replication role to start walsenderRole lacks REPLICATION
number of requested standby connections exceeds max_wal_sendersToo few sender slots
database system identifier differs between the primary and standbyThe standby is not a copy of this primary
can no longer access replication slot "..."Slot invalidated (lesson XIV-06)

The identifier mismatch deserves its own note, because it was produced by accident while preparing this part:

Service impact possiblea standby built the wrong way
$ tail /tmp/sb.log
FATAL:  database system identifier differs between the primary and standby
DETAIL:  The primary's identifier is 7678833575722270763, the standby's identifier is 7678833643757821995.

The data directory had been wiped and re-created by pg_basebackup while the old postmaster was still alive; on shutdown it rewrote pg_control with its own identifier, overwriting the one the backup had installed. The procedural lesson is small and absolute: stop the target cluster before replacing its data directory, and confirm it stopped.

What to take from this

  • pg_stat_replication has one row per connected standby. Zero rows is the alert.
  • pg_stat_wal_receiver uses written_lsn and flushed_lsn; there is no received_lsn.
  • A replication role needs the REPLICATION attribute and no table grants.
  • pg_hba.conf needs a line for the replication keyword; host all all does not cover it.
  • pg_basebackup -R writes the password in plain text. Replace it.
  • Stop a cluster before replacing its data directory, or you get an identifier mismatch.

Cross-course references

  • Linux for Production Sysadmins — Part VI (Processes) covers seeing the sender and receiver as operating-system processes, and Part XLII (Network Performance) covers measuring the link they depend on.
  • Observability for Production Sysadmins — Part LX (Network observability) covers the bandwidth and latency series that explain most lag before any database setting does.

Quiz

Knowledge check · 6 questions

  1. Q1. A standby cannot connect and the log says 'no pg_hba.conf entry for replication connection'. The file already contains 'host all all 10.0.0.0/24 scram-sha-256'. Why is that not enough?

  2. Q2. A monitoring check runs 'SELECT max(replay_lag) FROM pg_stat_replication' and alerts when the value exceeds a threshold. What does it fail to detect?

  3. Q3. A standby's PGDATA was replaced by pg_basebackup while its own postmaster was still running. On restart it reports 'database system identifier differs between the primary and standby'. What happened?

  4. Q4. What does the standby report back to the primary over the replication connection? Select all that apply.

  5. Q5. A replication role needs no SELECT privilege on any table, because it never reads through the SQL layer.

  6. Q6. Why can pg_stat_replication show a plausible-looking row for a standby that is actually dead, and what distinguishes the two?

Passing score: 75%. Answers are checked in this browser.