[dimitri]

Problem

#1199 fixed synchronized_standby_slots staying stale when a standby catches up after the primary has already converged to steady primary: MonitorFSM[]'s pos 302 forces that primary through apply_settings the moment the standby's reportedState goes catchingup -> secondary, but only when the primary's own row already reads reportedState == goalState == primary.

A narrower window was left open: a standby finishing catch-up while the primary itself is still mid-promotion (reportedState=wait_primary, goalState already primary) never gets the same treatment. In a three-node group started together, this produced a primary stuck applying a one-slot synchronized_standby_slots list while the monitor's own view already listed two standbys, observed in roughly 1 in 10 concurrent three-node startups.

Root cause

fsm_enable_sync_rep()'s WAIT_PRIMARY -> PRIMARY keeper transition (fsm_transition.c) is retried on every heartbeat until primary_standby_has_caught_up() returns true, and each retry re-fetches synchronized_standby_slots fresh via fsm_apply_settings(). Crucially, primary_standby_has_caught_up() checks local pg_stat_replication, not the monitor's own reportedState column — so this retry loop can succeed off a single caught-up standby's WAL position before that other standby's own catchingup -> secondary transition has actually landed on the monitor.

Since that transition is a one-time event per standby (pos 302 is gated on previouslyReportedState == catchingup, via the onDiskSnapshot mechanism from #1199), if it's processed while the primary's row still reads reportedState=wait_primary (mid-promotion), pos 302's original strict FSM_STATE(PRIMARY) gate simply doesn't match — and it can never retroactively fire again for that standby. The primary's own goalState (and, in the real keeper, its locally-applied synchronized_standby_slots GUC) is left stuck on whatever shorter list had already been fetched, indefinitely.

Fix

Widen pos 302's primaryNode gate from FSM_STATE(REPLICATION_STATE_PRIMARY) to a new FSM_PRIMARY_CONVERGED_OR_PROMOTING pattern: a NODE_STATE_TRANSITIONING pattern matching reportedState in {primary, wait_primary} with goalState already primary.

This specifically catches the primary in that exact mid-promotion window, without also matching a primary merely sitting at steady wait_primary with no promotion pending (goalState still wait_primary too — e.g. waiting on its first quorum secondary), which must keep resolving straight to primary on its own next heartbeat, undisturbed. Confirmed via node_active_protocol.sql's own ordinary one-standby startup scenario, which initially regressed with a naive reportedState-only widening and is now unaffected.

Deliberately excludes join_primary from the pattern's reportedStates, unlike the existing FSM_WAIT_OR_JOIN_PRIMARY_TRANSITIONING_TO_PRIMARY: no (JOIN_PRIMARY_STATE, APPLY_SETTINGS_STATE) transition exists in the keeper's own FSM table (fsm.c), so admitting it would assign a goalState the keeper has no transition function for. The (WAIT_PRIMARY_STATE, APPLY_SETTINGS_STATE) edge this fix now exercises already existed in the keeper's FSM table (a no-op transition) but was previously unreachable from the monitor side — check_fsm_reachability's own edge-count assertions confirm this (178 -> 179 total edges, and the fixture-based keeper_fsm_edges test shows that edge disappearing from its "keeper edge with no monitor counterpart" list).

Test

New regression test synchronized_standby_slots_promoting_primary.sql drives the exact interleaving deterministically through pgautofailover.node_active() calls (no timing/Docker dependency):

1. Both standbys are assigned catchingup concurrently while the primary is still steady wait_primary (an ordinary two-standby startup).
2. One standby (s1) finishes catching up first, which is what nudges the primary into promoting (goalState=primary assigned, reportedState still wait_primary — the mid-promotion window).
3. The second standby (s2) only reports finishing (catchingup -> secondary) while the primary is in that exact window.
4. Asserts the primary's goalState becomes apply_settings immediately afterwards, not left at primary.

Confirmed failing before this fix (goalState stays primary) and passing after (goalState becomes apply_settings).

Also re-ran the full existing suite (regress + isolation, under USE_ASSERT_CHECKING so AssertMonitorFSMWellFormed()'s reachability cross-check actually runs) to confirm no regression, including node_active_protocol.sql, guard_data_loss.sql, and the original synchronized_standby_slots.sql from #1199 — all pass.

Style: citus_indent --check passes; ci/banned.h.sh reports no banned API usage.