Skip to content

fix(search): make indexing converge and survive shutdown - #29487

Merged
zoom-ua merged 1 commit into
masterfrom
joshblum/hotpot-search-convergence
Aug 11, 2026
Merged

fix(search): make indexing converge and survive shutdown#29487
zoom-ua merged 1 commit into
masterfrom
joshblum/hotpot-search-convergence

Conversation

@zoom-ua

@zoom-ua zoom-ua commented Jul 29, 2026

Copy link
Copy Markdown
Collaborator

adapted from #29467

@mmaxim

@zoom-ua zoom-ua changed the title fix(search): make indexing converge and survive shutdown (draft) fix(search): make indexing converge and survive shutdown Jul 29, 2026
@zoom-ua
zoom-ua force-pushed the joshblum/hotpot-search-convergence branch from 1227be1 to e40b5cf Compare August 11, 2026 14:39
@zoom-ua zoom-ua changed the title (draft) fix(search): make indexing converge and survive shutdown fix(search): make indexing converge and survive shutdown Aug 11, 2026
@zoom-ua

zoom-ua commented Aug 11, 2026

Copy link
Copy Markdown
Collaborator Author
  1. Evicted dirty entries lost — Entries evicted from LRU cache before flush were permanently dropped; now tracked in pending set and written on next flush
  2. Failed writes dropped — Flush cleared dirty tracking before writing to disk; write failures left messages marked indexed with no tokens (unsearchable forever); now requeues failed writes for retry
  3. Deletes racing with flush — Direct-to-disk deletes raced with concurrent flush snapshots, resurrecting deleted entries; now queues tombstones that flush applies in order
  4. Metadata published ahead of tokens — Metadata (SeenIDs) written directly while tokens were still pending; convs read as complete with missing tokens; now routes all writes through overlay
  5. Reindexing refetches everything — Paged over full min..max range instead of missing IDs only; spent budget refetching already-indexed messages; now pages over missing IDs
  6. Deleted/expired messages block convergence — IDs the server won't return (deleted messages, retention policy expiry, gaps) held numMissing > 0 forever, causing infinite refetch loops; now marks unreturned IDs seen after successful fetch
  7. Stop/Start lifecycle races — Concurrent Stop/Start could replace store while old run still uses it; now serializes runs (Start waits for previous Stop to finish)
  8. Storage failures unreported — Add/Remove ops always completed callback successfully even when indexing failed, causing reindexConv to mark unindexed messages as done; now propagates errors via callback
  9. Clear() global side effects — Clearing one conv discarded all other convs' pending writes (ClearMemory is global, disk clear is per-conv); now flushes before clearing

@zoom-ua
zoom-ua force-pushed the joshblum/hotpot-search-convergence branch from e40b5cf to 4378de4 Compare August 11, 2026 15:53
@zoom-ua
zoom-ua merged commit f5dc0c2 into master Aug 11, 2026
1 check passed
@zoom-ua
zoom-ua deleted the joshblum/hotpot-search-convergence branch August 11, 2026 17:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant