You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 4c8affa
Browse filesBrowse the repository at this point in the historyBrowse files
docs(cluster_setup): cover synchronous replication and switchover
The cluster_setup page only described a single asynchronous standby: a
replication user, wal_level=replica, pg_basebackup -R, and a raw
pg_stat_replication dump. Nothing covered synchronous replication, how to
promote a standby, what has to be cleaned up afterwards, or how the
Oracle-compatible listener behaves on a standby.
Add four sections to the CN and EN pages:
- synchronous replication: both synchronous_standby_names notations, what
they turn sync_state into (sync vs quorum), and the lag columns;
- promotion drill: pg_ctl promote / pg_promote(), the read-only check, the
split-brain check, and pg_rewind to re-attach the old primary;
- per-node ivorysql.port consistency: the same instance answers with
compatible_mode=pg or oracle depending on which listener the client
reaches, so a standby that only sets port either fails to bind the
Oracle socket or serves the wrong semantics;
- replication monitoring: sync_priority, sync_state, and the lag columns.
The promotion section documents the trap that costs the most time: a
standby built with pg_basebackup -R inherits synchronous_standby_names, so
right after promotion every commit waits in wait_event_type=IPC,
wait_event=SyncRep and statement_timeout does not interrupt it. Clearing
the parameter needs two separate statements because ALTER SYSTEM cannot
run inside the same transaction as the reload.
The cluster built above is asynchronous streaming replication: the primary does not wait for a standby to confirm a commit. If you need stronger durability, enable synchronous replication. The cost is that **every commit waits until a standby has flushed the WAL to disk**, so transaction latency on the primary includes the standby's disk write and the network round trip. Append the following to `postgresql.conf` on the primary node (the ports follow the examples in this chapter: `port = 5532` and `ivorysql.port = 1621` on the primary):
`synchronous_commit = on` is the default, so the parameter that actually decides whether replication is synchronous is `synchronous_standby_names`: leave it empty and the cluster stays asynchronous; set it and commits wait for the standbys in the list. There are two notations, and **the value a standby reports in `pg_stat_replication.sync_state` depends on which one you use**:
193
+
194
+
. Named notation: write the standby's `application_name` (`'walreceiver'`), or the form with a priority, `FIRST 1 (walreceiver)`. The selected standby reports `sync_state = sync`.
To verify it, check `sync_state` on the primary: `sync` or `quorum` means synchronous replication is in effect; if it is still `async`, the setting was most likely never loaded (a missing reload or restart), or the standby's `application_name` does not match the name list. When every standby in the list is disconnected, write transactions on the primary stay parked in the synchronous replication wait — deal with this parameter before any maintenance or switchover, see the next section.
217
+
218
+
== Failover drill
219
+
220
+
Promoting a standby turns it into a writable primary. Both ways below are equivalent:
`pg_promote` returns `t` once the promotion has been requested and waited for, and `pg_is_in_recovery()` going from `true` to `false` means the promotion succeeded. Before promotion the standby is read-only and writes are rejected:
251
+
252
+
[source,text]
253
+
----
254
+
ERROR: cannot execute CREATE TABLE in a read-only transaction
255
+
----
256
+
257
+
[CAUTION]
258
+
A standby built with `pg_basebackup -R` **inherits** the primary's `synchronous_standby_names` and `synchronous_commit`. After promotion it is the primary, but it usually has no standby attached, so commits park in the synchronous replication wait. Measured with `statement_timeout` set to `8s`, the transaction was still waiting when sampled at the 5 second mark:
1132 | active | IPC | SyncRep | 773 | set statement_timeout='8s'; insert into sync_probe(id,note)
265
+
----
266
+
267
+
So the first thing to do after promotion is to clear the setting (or rewrite it with this node's real standby list) on the new primary:
268
+
269
+
[source,sql]
270
+
----
271
+
ALTER SYSTEM SET synchronous_standby_names = '';
272
+
SELECT pg_reload_conf();
273
+
----
274
+
275
+
Run the two statements separately; combining them in one `-c` with a semicolon fails with `ERROR: ALTER SYSTEM cannot run inside a transaction block`. After clearing it `sync_names=[]`, and the same `INSERT` returned `INSERT 0 1` immediately.
276
+
277
+
The old primary has no idea it is no longer the primary. Measured after the promotion it was still running as a primary, with no standby attached to it:
The old primary must already be shut down — it is still a writable primary right now, and rewinding while it runs leaves you with two primaries accepting writes. `pg_rewind` only moves the data page differences back, so after the rewind still re-check this node's `port`, `ivorysql.port` and `primary_conninfo` as described earlier in this chapter.
294
+
295
+
== Oracle-compatible port consistency across primary and standby
296
+
297
+
A single IvorySQL instance listens on two TCP ports at once: the PostgreSQL port `port` and the Oracle-compatible port `ivorysql.port`. The startup log shows both listener pairs:
298
+
299
+
[source,text]
300
+
----
301
+
LOG: listening on IPv4 address "127.0.0.1", port 5532
302
+
LOG: listening on IPv4 address "127.0.0.1", port 1621
303
+
LOG: listening on Unix socket "/tmp/.s.PGSQL.5532"
304
+
LOG: listening on Unix socket "/tmp/.s.PGSQL.1621"
305
+
----
306
+
307
+
**The listener a session connects to decides the compatibility mode it sees.** Query it directly on one instance:
308
+
309
+
[source,sql]
310
+
----
311
+
SHOW port;
312
+
SHOW ivorysql.port;
313
+
SHOW ivorysql.compatible_mode;
314
+
----
315
+
316
+
Measured values (same instance, same data, only the connection port differs):
317
+
318
+
[cols="1,1,1,1,1"]
319
+
|===
320
+
| Node | port | ivorysql.port | Connected to | compatible_mode
321
+
| Primary | 5532 | 1621 | 5532 | pg
322
+
| Standby | 5533 | 1622 | 5533 | pg
323
+
| Standby | 5533 | 1622 | 1622 | oracle
324
+
|===
325
+
326
+
In other words `ivorysql.port` is not an alias of `port` but a separate listener: connecting to 5533 gives `pg`, connecting to 1622 gives `oracle`. That is why **`ivorysql.port` must be configured on every node**: when you edit `postgresql.conf` on the standby as described earlier, change `port` and `ivorysql.port` together. Change only `port` and leave `ivorysql.port` at 1521, and an application that connects to the standby on 1521 either lands on whichever other instance already owns that port or the node simply refuses to start:
327
+
328
+
[source,text]
329
+
----
330
+
LOG: listening on IPv4 address "127.0.0.1", port 5534
331
+
LOG: could not bind IPv4 address "127.0.0.1": Address already in use
332
+
HINT: Is another postmaster already running on port 1621? If not, wait a few seconds and retry.
333
+
WARNING: could not create listen socket for "localhost"
334
+
FATAL: could not create any Oracle TCP/IP sockets
335
+
LOG: database system is shut down
336
+
----
337
+
338
+
(`pg_ctl` also reports `pg_ctl: could not start server`.) You can reproduce this on a single machine: while the primary still owns 1621, start a second instance from another data directory with `-p 5534 -c ivorysql.port=1621`. The same applies on the application side: an application that needs Oracle-compatible semantics has to connect to `ivorysql.port`, and the value of that port comes from the configuration of **the node it actually connects to**, so the `ivorysql.port` on a standby must be the port that standby really listens on.
339
+
340
+
== Replication monitoring
341
+
342
+
`pg_stat_replication` on the primary has one row per standby. Day-to-day monitoring only needs these columns:
| `application_name` | The name the standby reports, `walreceiver` by default; named synchronous replication picks the synchronous standby by it
356
+
| `state` | `streaming` means WAL is being streamed in real time
357
+
| `sync_state` | `async` (asynchronous), `sync` (synchronous standby under named notation), `quorum` (synchronous standby under quorum notation)
358
+
| `sync_priority` | Priority within the name list, lower numbers first
359
+
| `sent_lsn` | WAL position the primary has sent to this standby
360
+
| `write_lsn` | WAL position this standby has written to disk
361
+
| `flush_lsn` | WAL position this standby has flushed, which is what synchronous replication waits for
362
+
| `replay_lsn` | WAL position this standby has replayed; reads on the standby see data up to here
363
+
| `write_lag` / `flush_lag` / `replay_lag` | How far behind `sent_lsn` those three positions are, as time intervals
364
+
| `reply_time` | When this standby last reported its status
365
+
|===
366
+
367
+
How to read it: `replay_lsn` behind `sent_lsn` means WAL is still waiting to be replayed, so reads on the standby may return old data; `write_lag`, `flush_lag` or `replay_lag` empty means no new WAL needed processing during the last status interval; `sync_state` only `async` means replication is still asynchronous and commits on the primary do not wait for a standby.
368
+
369
+
A standby is read-only because `transaction_read_only = on`, and every write is rejected (see the read-only example in the previous section for the exact error). When you need to write, promote it as described in "Failover drill" instead of trying to turn read-only off on the standby.
0 commit comments