fix(remote): earn a statistics re-dump at a fifth, not a half - #232
Merged
Conversation
Size Drift decided a snapshot was still current until a table's reltuples moved by half. That is coarser than the signal it reads: pg_class.reltuples is rewritten when autovacuum analyzes the table, which by default happens once it has changed by 10%, so four fifths of the available resolution went unused. It also let a table sit at 1.5x its snapshot size while every query against it was costed a third light. Move the ratio to 0.2 — two ticks of the underlying signal, and small enough that the growth/shrink asymmetry in |now - before| / before stops mattering. Co-Authored-By: Claude <noreply@anthropic.com>
There was a problem hiding this comment.

Query Doctor — 6 successful checks
More details via MCP → get_ci_run({ runId: "019ffd57-3a69-7252-aba4-ab993ff5800c" }) · view run · docs
3 queries read against main on assumed statistics of 10,000,000 rows per table. Sync production stats for costs measured against your real data.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Goal
Queries should be costed against statistics that match the database they run on. The analyzer keeps a project's stored statistics current by re-dumping when the source has moved; this changes how far it has to move before that happens.
What
Before: a table had to grow by half, or shrink to half, before the analyzer re-dumped the source's statistics. Between re-dumps a table could sit at 1.5x its recorded size, and every query against it was costed a third light.
After: a fifth of movement is enough. A table at 1.25x its recorded size now earns a re-dump instead of being called current.
The re-dump itself is unchanged, as is the 24-hour floor that fires when nothing has drifted.
How
src/remote/stats-drift.tsis the whole change:DEFAULT_SIZE_DRIFT_RATIOgoes from 0.5 to 0.2.The number is bounded on both sides. Underneath, the check reads
pg_class.reltuples, which Postgres only rewrites when autovacuum analyzes the table, by default once it has changed byautovacuum_analyze_scale_factor, 10%. A threshold below that reads sampling noise rather than data. Above, the threshold is how wrong a cost may be before it is worth acting on, and 0.5 tolerated a cost a third light.0.2 also makes the growth and shrink sides close to even. The comparison is
|now - before| / before, so 0.5 fired on 1.5x growth but needed a 2x shrink; at 0.2 the two are 1.25x and 1.2x.Tests
src/remote/stats-drift.test.ts: a 1.25x move drifts, a 1.1x move does not. The near-miss case that names the closest table sits at 15% and 5%, either side of the new threshold.Full suite passes (443 tests, 43 files),
npm run typecheckclean.