fix(api): paginate sample history in SQL - #1167
Open
gaurav02081 wants to merge 2 commits into
Open
Conversation
GET /samples/{id}/history applied limit/offset in Python after building an
entry for every result in the sample's history, so the page size had no
effect on the work a request did. On production this hit the gateway
timeout (504 after ~60-80s for ?limit=5).
The page is now cut in SQL, so the follow-up queries (result files with
their two nested joinedloads, tests, run timestamps) see one page rather
than the whole history. Ordering gains regression_test_id as a tiebreaker
so a row can't shift between pages when several regression tests share a
run.
?status can't move into SQL with it: status comes from
derive_sample_status, which needs result files and expected outputs, and
mod_api.services.status is meant to be the only place that derivation
lives. That path instead scans a bounded window of recent results and
reports pagination.truncated when a sample has more history than the
window covered, following the convention the Page schema already
documents.
Adds ?regression_test_id so limit can mean runs of one regression test
rather than rows spread across every test on the sample. Without it a
page of N covers only about N / (tests on the sample) runs of the test a
caller cares about, which made history-based verdicts wrong rather than
just slow: a test that passed outside that short window looked as though
it had never passed.
The spec also pointed ?status at the RunStatus enum
(queued/running/pass/fail/canceled/incomplete); it now describes the
per-sample statuses the endpoint actually accepts.
Fixes CCExtractor#1161
gaurav02081
requested review from
canihavesomecoffee and
thealphadollar
as code owners
August 9, 2026 18:49
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.



Fixes #1161.
The bug
get_sample_historyappliedlimit/offsetin Python after building an entryfor every result in the sample's history:
So the page size had no effect on the work a request did. On production that
was 504 after ~60-80s for
?limit=5. It doesn't reproduce on a small devdatabase, which is why it got through review.
The batching that was already there (
batch_get_run_data,_preload_expected_outputs) was doing its job — it avoided N+1 — but itbatched over the whole history instead of over the page being returned.
The fix
Pagination moves into SQL.
total = query.count()plus.offset(offset).limit(limit), and the Python slice is gone. Everythingdownstream — result files with their two nested joinedloads, tests,
timestamps — now scales with the page. The loading is factored into
_build_history_entries(results, ...), which is keyed entirely off theresults handed to it, with a docstring saying callers must pass the page and
not the whole history.
Ordering gains a tiebreaker.
ORDER BY Test.id DESCalone was ambiguousacross the several rows a run contributes when a sample has more than one
regression test. That didn't matter while everything was ordered once in
memory; with real
LIMIT/OFFSETit lets a row repeat on one page and vanishfrom another. Ordering is now
(Test.id DESC, TestResult.regression_test_id ASC).?statusgets a bounded scan. This is the complication the issue callsout, and I went with option 2 over option 1 deliberately: status comes from
derive_sample_status, which needs the result files and the expected outputs,and
mod_api/services/status.pystates it is the single source of truth andthat routes must not inline their own derivation. Reproducing that in SQL
would put the derivation in two places, and the day they disagree is a worse
bug than this one. So that path scans a bounded window of recent results
(
_HISTORY_STATUS_SCAN_LIMIT = 1000) and setspagination.truncatedwhen thesample has more history than the window covered — the convention the
Pageschema already documents ("Present and true when the result set was capped by
an internal safety limit (e.g. status-filter on runs)"). The unfiltered path,
the one that actually 504s, is fully paginated.
New
?regression_test_idfilter, for the semantics problem in thefollow-up comment. Applied in SQL alongside the sample's regression tests, so
limitmeans that many runs of one test rather than that many rows spreadacross every test on the sample. This is the part that was returning wrong
answers rather than slow ones: with 10 tests on a sample,
?limit=20coveredabout 2 runs of the test you asked about, so a pass 8 runs back fell outside a
window the caller couldn't tell was short. A regression test belonging to a
different sample is a 400 rather than a silently empty page — an empty page
would read as "this test has no history", which isn't true. It composes with
?status, so a status scan on a single test now reaches ~1000 runs deepinstead of ~100.
Spec
The history endpoint's
?statuspointed at theRunStatusenum(
queued/running/pass/fail/canceled/incomplete), but the endpoint acceptspass/fail/missing_output/not_started. AddedSampleHistoryStatuswith theright values, plus
RegressionTestIdFilter, and documented the truncationbehaviour on the path. Worth a look since schemathesis fuzzes off this spec.
Tests
Six tests in
tests/api/test_routes_samples.py. The one that matters most istest_get_sample_history_loads_only_the_page: it builds 8 runs, requests?limit=2, and captures the emitted SQL to assert thetest_result_fileselect binds no more test ids than the page holds.
I ran the new tests against the unfixed route to check they earn their place:
9 test ids for a 2-row page — the bug, caught directly. The rest cover SQL
pagination with non-overlapping pages, the
regression_test_idfilter and itstwo rejection cases, status filtering, and the truncated flag (scan limit
patched down to 2).
tests.apipasses in full (264 tests). isort, pydocstyle, pycodestyle andmypy are clean on the changed files.
Notes for review
_HISTORY_STATUS_SCAN_LIMIT = 1000is a guess — I don't have productiontimings for the status path. Happy to move it once someone with access can
measure what a 1000-row scan actually costs there.
bounded scan turns out too coarse, the honest version of that is teaching
the status service to emit a SQL expression, so the derivation stays in one
place — bigger than this PR should be.