fix(query-db-collection): preserve in-memory row ownership when hydrating from persisted metadata#1483
Open
goatrenterguy wants to merge 2 commits intoTanStack:mainfrom
Open
Conversation
…ting from persisted metadata
`getHydratedOwnedRowsForQueryBaseline` and the `scanPersisted` path in
`loadPersistedBaselineForQuery` used to overwrite `rowToQueries[rowKey]`
with whatever the persisted metadata held, silently evicting any query
that had registered ownership in-memory only. `queryToRows` was left
untouched, so the two maps desynced and `cleanupQueryInternal` could
later delete rows still needed by an active overlapping query.
In-memory and persisted ownership can legitimately diverge: when
`applySuccessfulResult` writes a brand-new row via `ctx.write({ type:
'insert', value })` without a `metadata` field, the insert branch in
`packages/db/src/collection/sync.ts` unconditionally queues
`{ type: 'delete' }` on the same transaction's `rowMetadataWrites`,
clobbering the earlier `setPersistedOwners({A})` call in the same
transaction. After commit, the in-memory map has `{A}` but persisted
metadata is empty. The next query that hits the hydration path then
stomps `rowToQueries[rowKey]` with the stale persisted owners.
Fix: merge persisted owners into the existing in-memory
`rowToQueries[rowKey]` instead of overwriting it, in both hydration
paths.
Added a regression test under `Query Garbage Collection` that
reproduces the scenario end-to-end (fresh-row insert by A, overlap
on the existing row by B, a new empty-result query C forcing the
hydration path, then B teardown) and asserts the shared row survives
when A is still subscribed. Fails without the fix, passes with it.
More templates
@tanstack/angular-db
@tanstack/browser-db-sqlite-persistence
@tanstack/capacitor-db-sqlite-persistence
@tanstack/cloudflare-durable-objects-db-sqlite-persistence
@tanstack/db
@tanstack/db-ivm
@tanstack/db-sqlite-persistence-core
@tanstack/electric-db-collection
@tanstack/electron-db-sqlite-persistence
@tanstack/expo-db-sqlite-persistence
@tanstack/node-db-sqlite-persistence
@tanstack/offline-transactions
@tanstack/powersync-db-collection
@tanstack/query-db-collection
@tanstack/react-db
@tanstack/react-native-db-sqlite-persistence
@tanstack/rxdb-db-collection
@tanstack/solid-db
@tanstack/svelte-db
@tanstack/tauri-db-sqlite-persistence
@tanstack/trailbase-db-collection
@tanstack/vue-db
commit: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes a row-ownership desync in
@tanstack/query-db-collectionthat causes rows to be incorrectly deleted fromsyncedDatawhen multiple overlapping on-demand live queries are active and one of them unmounts.Observed in production as: after a mutation inserts a row through an
onInserthandler, opening and then closing a detail panel (which unmounts a sibling live query) causes rows still covered by a subscribed broader live query to disappear from the UI. The broader query's cache is intact but the collection'ssyncedDatahas been truncated.Root cause
getHydratedOwnedRowsForQueryBaseline(and thescanPersistedbranch inloadPersistedBaselineForQuery) seed ownership for a brand-new query by doing:where
ownerscomes fromgetPersistedOwners(rowKey). This overwrites the in-memory entry with whatever persisted metadata currently holds, while leavingqueryToRowsfor other queries untouched. When in-memory ownership and persisted ownership are out of sync, any active query whose entry is only in-memory gets silently evicted fromrowToQueries, while itsqueryToRows[…]entry still claims the row. Later,cleanupQueryInternal(which consultsrowToQueriesas truth) seesnextOwners.size === 0for the shared row and deletes it, even though another subscribed query still covers it.Why in-memory and persisted can diverge
applySuccessfulResult'snewItemsMaploop for a fresh row runs this sequence inside a singlebegin()/commit():setPersistedOwners(key, { A })— queues{ type: 'set', value: { queryCollection: { owners: { A: true } } } }on the pending transaction'srowMetadataWrites.addRow(key, A)— updates the in-memory maps.ctx.write({ type: 'insert', value: newItem })— the insert branch inpackages/db/src/collection/sync.ts(~L176-186) unconditionally setsrowMetadataWrites[key] = { type: 'delete' }, overwriting step 1 on the same key in the same transaction.On commit, the delete wins. Net state after the first query:
rowToQueries[key]={ A }syncedMetadata[key]= undefinedWhen a second query subsequently touches the same row via the update path (existing row → no insert clobber), its
setPersistedOwnerssurvives but writes only{ B }becausegetPersistedOwnersreturned empty. Net state:rowToQueries[key]={ A, B }syncedMetadata[key].queryCollection.owners={ B }A third query entering
getHydratedOwnedRowsForQueryBaselinethen overwritesrowToQueries[key]back to{ B }, dropping A, and the teardown path deletes the row.Fix
Merge persisted owners into the existing in-memory
rowToQueriesentry instead of overwriting it, in both hydration sites:Narrowest intervention: hydration now tolerates an out-of-date persisted record without dropping in-memory ownership that active queries registered via
addRow.Test
Added a regression test under
Query Garbage Collection:hydration must not strip active in-memory query owners when persisted metadata is out of dateIt builds the scenario deterministically:
applySuccessfulResulttakes the update path and persists{ B }(A is never persisted).Asserts the row is still in
syncedDataafterward. Fails without the patch (row is deleted), passes with it.Full suite: 192/192 passing.
Out of scope
This PR fixes the hydration side. The insert-clobber in
sync.tsthat creates the divergence in the first place is a broader change with framework-wide implications (it would alter how manual sync backends interact with row metadata across transactions). Happy to open a separate discussion on whether the insert branch should preserve a pending{ type: 'set' }in the same transaction rather than unconditionally writing{ type: 'delete' }— but it's not required for this bug.Test plan
main, passes with this changepnpm --filter @tanstack/query-db-collection testpasses (192/192)