Skip to content

WPB-22969: migrate password-reset to PostGreSQL - #5412

Merged
blackheaven merged 2 commits into
developfrom
gdifolco/WPB-22969-migration-postgres-password-reset
Oct 1, 2026
Merged

blackheaven merged 2 commits into
developfrom
gdifolco/WPB-22969-migration-postgres-password-reset

Conversation

@blackheaven

@blackheaven blackheaven commented Aug 4, 2026 •

Copy link
Copy Markdown
Contributor

https://wearezeta.atlassian.net/browse/WPB-22969

Checklist

  • Add a new entry in an appropriate subdirectory of changelog.d
  • Read and follow the PR guidelines

@blackheaven
blackheaven requested review from a team as code owners August 4, 2026 11:48
@zebot zebot added the ok-to-test Approved for running tests in CI, overrides not-ok-to-test if both labels exist label Aug 4, 2026
@blackheaven
blackheaven force-pushed the gdifolco/WPB-22969-migration-postgres-password-reset branch from 0aff1b4 to 794cf34 Compare August 4, 2026 12:33
@blackheaven
blackheaven force-pushed the gdifolco/WPB-22969-migration-postgres-password-reset branch 2 times, most recently from 7c2a589 to 2c147ec Compare September 11, 2026 17:49
@blackheaven
blackheaven force-pushed the gdifolco/WPB-22969-migration-postgres-password-reset branch from 2c147ec to 0c9e296 Compare September 11, 2026 18:24
@blackheaven
blackheaven force-pushed the gdifolco/WPB-22969-migration-postgres-password-reset branch from 0c9e296 to b4ae5b7 Compare September 24, 2026 07:23
Comment on lines +53 to +54
runCodensity (startDynamicBackend backend (conf "migration-to-postgresql" True)) $ \_ ->
waitForMigration domain counterName

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could also check code here.

Comment on lines +29 to +32
-- | Drives the password-reset store through the full cutover lifecycle
-- (cassandra -> migration-to-postgresql -> postgresql). A reset code written to
-- Cassandra before migration must be served from Postgres after the cutover and
-- still complete the reset, proving the row was backfilled.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should try to do a write in in every step and try to read it in the next step again. This way we're also testing the writes from all 3 interpreters.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also we need some tests to make sure edge cases of TTL are dealt with correctly.

CREATE TABLE IF NOT EXISTS password_reset (
key text PRIMARY KEY,
code text NOT NULL,
"user" uuid NOT NULL,

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"user" uuid NOT NULL,
user_id uuid NOT NULL,

This way it is easier to detect issues in queries. I'm trying to avoid anything called "user" in postgresql because they don't error until parsing the results.

Comment on lines +56 to +60
-- | A 6-digit, zero-padded code (mirrors the Cassandra interpreter).
genPhoneCode :: (Member (Embed IO) r) => Sem r PasswordResetCode
genPhoneCode =
PasswordResetCode . unsafeFromText . pack . printf "%06d"
<$> embed @IO (randIntegerZeroToNMinusOne 1000000)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an unused function, feel free to delete it.

@blackheaven
blackheaven force-pushed the gdifolco/WPB-22969-migration-postgres-password-reset branch from f2f4c17 to 518acee Compare October 1, 2026 09:41
@blackheaven

Copy link
Copy Markdown
Contributor Author

Addressed the review feedback (rebased onto current develop, squashed to one commit 518acee061):

  • Column rename: password_reset."user" → user_id in the migration, the Hasql statements, and postgres-schema.sql.
  • genPhoneCode: removed from the Postgres module's export list (kept as a non-exported local so GeneratePhoneCode stays total); the dual-write interpreter now delegates phone-code generation to the Cassandra interpreter. Note the whole phone-reset path is dead (no production callsite, resetPasswordImpl rejects phone identity) — removing it entirely (effect constructor + both copies) could be a follow-up cutover if preferred.
  • Integration test rewritten: each phase now writes with a fresh user (reset keys are per-user; a second reset is a silent no-op) and the next phase reads it back — covering writes of all three interpreters. The Postgres phase also checks the worker backfilled the Cassandra-only row, completes the dual-write flow, and exercises the retry edge cases: wrong codes decrement (3→2→1), the third wrong attempt deletes the row, after which even the correct code is rejected and the password is unchanged.
  • Also fixed a tuple-arity compile error in BackgroundWorker.hs (the cleanup 10-way Concurrently tuple had a 9-slot constructor) that was failing wire-server-compile-nix, and a copy-paste in the developer docs cutover example (passwordReset: cassandra → postgresql in step 3).

Known deviation worth a look: the backfill worker upserts rows without the per-row exclusive migration lock that the user/domainRegistration/teamFeatures/conversationCodes migrations use, so a row consumed (reset completed / invalidated) between the worker's Cassandra read and Postgres write could be re-inserted with a fresh expiry. Narrow window, but it deviates from the sibling migrations' locking pattern — happy to add withExclusiveMigrationLockAndTimeout + dual-write lock serialization if you'd rather match them.

@akshaymankar akshaymankar left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

genPhoneCode: removed from the Postgres module's export list (kept as a non-exported local so GeneratePhoneCode stays total); the dual-write interpreter now delegates phone-code generation to the Cassandra interpreter. Note the whole phone-reset path is dead (no production callsite, resetPasswordImpl rejects phone identity) — removing it entirely (effect constructor + both copies) could be a follow-up cutover if preferred.

I meant we can delete it from the effect itself since there are no call sites.
But it was a nit anyway, so its ok if you merge this without.

Comment on lines +72 to +74
initiateReset domain >>= \u3' -> do
waitForMigration domain counterName
pure u3'

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unless I'm missing something, why does the waiting need to be explicitly bound? This is the same right?:

Suggested change
initiateReset domain >>= \u3' -> do
waitForMigration domain counterName
pure u3'
u3 <- initiateReset domain
waitForMigration domain counterName
pure u3

@blackheaven
blackheaven force-pushed the gdifolco/WPB-22969-migration-postgres-password-reset branch from 9750ae8 to 40f3a18 Compare October 1, 2026 14:30
@blackheaven
blackheaven merged commit 9d883c3 into develop Oct 1, 2026
10 checks passed
@blackheaven
blackheaven deleted the gdifolco/WPB-22969-migration-postgres-password-reset branch October 1, 2026 15:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ok-to-test Approved for running tests in CI, overrides not-ok-to-test if both labels exist

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants