Skip to content

RFC: Mutation log reconciliation for optimistic writes#1630

Draft
KyleAMathews wants to merge 32 commits into
mainfrom
rfc-transactions-sync-state
Draft

RFC: Mutation log reconciliation for optimistic writes#1630
KyleAMathews wants to merge 32 commits into
mainfrom
rfc-transactions-sync-state

Conversation

@KyleAMathews

@KyleAMathews KyleAMathews commented Jun 30, 2026

Copy link
Copy Markdown
Collaborator

Summary

Implements Phase 1 of Mutation Log reconciliation for optimistic writes. Committed sync/base updates now apply immediately while unsettled optimistic mutations remain projected in visible collection state, improving reconciliation stability for collections, subscribeChanges, and live-query consumers.

Root Cause

TanStack DB previously treated persisting local transactions as a reason to delay normal committed sync application. That avoided some flicker, but it also meant synced/base state could lag behind while optimistic mutations were active, making materialized visible state and change events depend on delayed-sync branches instead of a single reconciliation model.

Approach

This PR shifts collection state toward the invariant:

authoritative synced/base state
+ unsettled optimistic mutations owned by the collection
= visible collection state

Key implementation details:

  • Apply committed sync/base transactions immediately in CollectionStateManager.commitPendingTransactions().
  • Capture previous visible state before sync/base mutation, then diff against final visible state after active optimistic mutations are projected.
  • Keep transaction state as the lifecycle source of truth; mutations derive from their owning transaction.
  • Preserve virtual-property transitions like $synced: false -> true when they are part of subscribed visible values.
  • Use keyed authoritative sync payloads in tests; this intentionally does not infer keys or server identity from matching row contents.

Key Invariants

  • A persisting transaction must not block unrelated committed sync/base rows from becoming visible.
  • An active optimistic mutation must remain visible over newer base state until the owning transaction settles and reconciliation removes it.
  • Change events are emitted from previous visible state to next visible state, not from raw synced/base deltas alone.
  • Identical sync confirmation should not produce duplicate user-data insert/update events, but virtual-only confirmation updates should still be emitted where materialized.
  • Sync payloads must provide normal keys; this PR does not add key-inference fallbacks or server-key matching heuristics.

Non-goals

This is intentionally limited to the Phase 1 internal reconciliation slice. It does not add:

  • public write-status APIs like $hasPendingWrites, $writeStatus, tx.when(...), or db.mutations
  • transport confirmation states like accepted or observed
  • operation targeting, base-only query modes, or sync batch API redesigns
  • server-key/stable-identity matching heuristics
  • offline durability changes

Trade-offs

The implementation keeps the existing mutation/transaction primitives instead of introducing a new operation lifecycle. That keeps Phase 1 smaller and aligned with current public vocabulary, but it leaves future public mutation-log/query APIs as follow-up work rather than mixing them into this reconciliation change.

Verification

pnpm --filter @tanstack/db exec tsc --noEmit
pnpm --filter @tanstack/db exec vitest run tests/collection-subscribe-changes.test.ts tests/collection.test.ts --pool-options.threads.maxThreads=2
pnpm --filter @tanstack/db test

Local validation during prep:

  • focused collection / subscribeChanges tests passed
  • full @tanstack/db suite passed before final prep-review fixes

Files changed

  • packages/db/src/collection/state.ts — immediate sync/base reconciliation, optimistic projection, visible-state diffing, cleanup/confirmation handling.
  • packages/db/src/collection/sync.ts — small sync write adjustment for keyed authoritative payload handling.
  • packages/db/tests/collection.test.ts — direct collection reconciliation regressions.
  • packages/db/tests/collection-subscribe-changes.test.ts — stable change-event, metadata, virtual prop, and confirmation regressions.
  • packages/db/tests/query/live-query-collection.test.ts and packages/db/tests/query/query-while-syncing.test.ts — live-query behavior under immediate-base reconciliation.
  • docs/rfcs/2026-06-25-mutation-log-reconciliation.md — RFC for the mutation-log reconciliation direction and future/non-Phase-1 boundaries.
  • docs/superpowers/plans/2026-06-29-mutation-log-reconciliation.md — implementation plan/history for this slice.
  • .changeset/mutation-log-reconciliation.md — patch changeset for @tanstack/db.

@coderabbitai

coderabbitai Bot commented Jun 30, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: a84fdf36-7757-49c6-8d95-7cdd882ea681

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch rfc-transactions-sync-state

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

KyleAMathews and others added 2 commits June 30, 2026 13:57
@pkg-pr-new

pkg-pr-new Bot commented Jun 30, 2026

Copy link
Copy Markdown
More templates

@tanstack/angular-db

npm i https://pkg.pr.new/@tanstack/angular-db@1630

@tanstack/browser-db-sqlite-persistence

npm i https://pkg.pr.new/@tanstack/browser-db-sqlite-persistence@1630

@tanstack/capacitor-db-sqlite-persistence

npm i https://pkg.pr.new/@tanstack/capacitor-db-sqlite-persistence@1630

@tanstack/cloudflare-durable-objects-db-sqlite-persistence

npm i https://pkg.pr.new/@tanstack/cloudflare-durable-objects-db-sqlite-persistence@1630

@tanstack/db

npm i https://pkg.pr.new/@tanstack/db@1630

@tanstack/db-ivm

npm i https://pkg.pr.new/@tanstack/db-ivm@1630

@tanstack/db-sqlite-persistence-core

npm i https://pkg.pr.new/@tanstack/db-sqlite-persistence-core@1630

@tanstack/electric-db-collection

npm i https://pkg.pr.new/@tanstack/electric-db-collection@1630

@tanstack/electron-db-sqlite-persistence

npm i https://pkg.pr.new/@tanstack/electron-db-sqlite-persistence@1630

@tanstack/expo-db-sqlite-persistence

npm i https://pkg.pr.new/@tanstack/expo-db-sqlite-persistence@1630

@tanstack/node-db-sqlite-persistence

npm i https://pkg.pr.new/@tanstack/node-db-sqlite-persistence@1630

@tanstack/offline-transactions

npm i https://pkg.pr.new/@tanstack/offline-transactions@1630

@tanstack/powersync-db-collection

npm i https://pkg.pr.new/@tanstack/powersync-db-collection@1630

@tanstack/query-db-collection

npm i https://pkg.pr.new/@tanstack/query-db-collection@1630

@tanstack/react-db

npm i https://pkg.pr.new/@tanstack/react-db@1630

@tanstack/react-native-db-sqlite-persistence

npm i https://pkg.pr.new/@tanstack/react-native-db-sqlite-persistence@1630

@tanstack/rxdb-db-collection

npm i https://pkg.pr.new/@tanstack/rxdb-db-collection@1630

@tanstack/solid-db

npm i https://pkg.pr.new/@tanstack/solid-db@1630

@tanstack/svelte-db

npm i https://pkg.pr.new/@tanstack/svelte-db@1630

@tanstack/tauri-db-sqlite-persistence

npm i https://pkg.pr.new/@tanstack/tauri-db-sqlite-persistence@1630

@tanstack/trailbase-db-collection

npm i https://pkg.pr.new/@tanstack/trailbase-db-collection@1630

@tanstack/vue-db

npm i https://pkg.pr.new/@tanstack/vue-db@1630

commit: 0645399

@github-actions

github-actions Bot commented Jun 30, 2026

Copy link
Copy Markdown
Contributor

Size Change: +266 B (+0.21%)

Total Size: 125 kB

📦 View Changed
Filename Size Change
packages/db/dist/esm/collection/state.js 5.59 kB +265 B (+4.98%) 🔍
packages/db/dist/esm/collection/sync.js 2.89 kB +1 B (+0.03%)
ℹ️ View Unchanged
Filename Size
packages/db/dist/esm/collection/change-events.js 1.43 kB
packages/db/dist/esm/collection/changes.js 1.38 kB
packages/db/dist/esm/collection/cleanup-queue.js 810 B
packages/db/dist/esm/collection/events.js 434 B
packages/db/dist/esm/collection/index.js 3.62 kB
packages/db/dist/esm/collection/indexes.js 1.99 kB
packages/db/dist/esm/collection/lifecycle.js 1.69 kB
packages/db/dist/esm/collection/mutations.js 2.47 kB
packages/db/dist/esm/collection/subscription.js 3.74 kB
packages/db/dist/esm/collection/transaction-metadata.js 144 B
packages/db/dist/esm/deferred.js 207 B
packages/db/dist/esm/errors.js 5.1 kB
packages/db/dist/esm/event-emitter.js 748 B
packages/db/dist/esm/index.js 3.1 kB
packages/db/dist/esm/indexes/auto-index.js 829 B
packages/db/dist/esm/indexes/base-index.js 767 B
packages/db/dist/esm/indexes/basic-index.js 2.06 kB
packages/db/dist/esm/indexes/btree-index.js 2.19 kB
packages/db/dist/esm/indexes/index-registry.js 820 B
packages/db/dist/esm/indexes/reverse-index.js 557 B
packages/db/dist/esm/local-only.js 916 B
packages/db/dist/esm/local-storage.js 2.12 kB
packages/db/dist/esm/optimistic-action.js 359 B
packages/db/dist/esm/paced-mutations.js 496 B
packages/db/dist/esm/proxy.js 3.75 kB
packages/db/dist/esm/query/builder/functions.js 1.47 kB
packages/db/dist/esm/query/builder/index.js 5.84 kB
packages/db/dist/esm/query/builder/ref-proxy.js 1.24 kB
packages/db/dist/esm/query/compiler/evaluators.js 1.89 kB
packages/db/dist/esm/query/compiler/expressions.js 430 B
packages/db/dist/esm/query/compiler/group-by.js 3.56 kB
packages/db/dist/esm/query/compiler/index.js 6.67 kB
packages/db/dist/esm/query/compiler/joins.js 2.5 kB
packages/db/dist/esm/query/compiler/lazy-targets.js 923 B
packages/db/dist/esm/query/compiler/order-by.js 1.74 kB
packages/db/dist/esm/query/compiler/select.js 1.53 kB
packages/db/dist/esm/query/effect.js 4.77 kB
packages/db/dist/esm/query/expression-helpers.js 1.43 kB
packages/db/dist/esm/query/ir.js 1.25 kB
packages/db/dist/esm/query/live-query-collection.js 360 B
packages/db/dist/esm/query/live/collection-config-builder.js 9.1 kB
packages/db/dist/esm/query/live/collection-registry.js 264 B
packages/db/dist/esm/query/live/collection-subscriber.js 1.93 kB
packages/db/dist/esm/query/live/internal.js 145 B
packages/db/dist/esm/query/live/utils.js 1.81 kB
packages/db/dist/esm/query/optimizer.js 2.92 kB
packages/db/dist/esm/query/predicate-utils.js 2.97 kB
packages/db/dist/esm/query/query-once.js 359 B
packages/db/dist/esm/query/subset-dedupe.js 960 B
packages/db/dist/esm/scheduler.js 1.3 kB
packages/db/dist/esm/SortedMap.js 1.3 kB
packages/db/dist/esm/strategies/debounceStrategy.js 247 B
packages/db/dist/esm/strategies/queueStrategy.js 428 B
packages/db/dist/esm/strategies/throttleStrategy.js 246 B
packages/db/dist/esm/transactions.js 3.03 kB
packages/db/dist/esm/utils.js 927 B
packages/db/dist/esm/utils/array-utils.js 273 B
packages/db/dist/esm/utils/browser-polyfills.js 304 B
packages/db/dist/esm/utils/btree.js 5.61 kB
packages/db/dist/esm/utils/comparison.js 1.11 kB
packages/db/dist/esm/utils/cursor.js 457 B
packages/db/dist/esm/utils/index-optimization.js 2.39 kB
packages/db/dist/esm/utils/type-guards.js 157 B
packages/db/dist/esm/utils/uuid.js 449 B
packages/db/dist/esm/virtual-props.js 360 B

compressed-size-action::db-package-size

@github-actions

Copy link
Copy Markdown
Contributor

Size Change: 0 B

Total Size: 4.26 kB

ℹ️ View Unchanged
Filename Size
packages/react-db/dist/esm/index.js 249 B
packages/react-db/dist/esm/useLiveInfiniteQuery.js 1.32 kB
packages/react-db/dist/esm/useLiveQuery.js 1.37 kB
packages/react-db/dist/esm/useLiveQueryEffect.js 355 B
packages/react-db/dist/esm/useLiveSuspenseQuery.js 567 B
packages/react-db/dist/esm/usePacedMutations.js 401 B

compressed-size-action::react-db-package-size

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant