Skip to content

feat(storage): speculation path set store - #501

Open
behinddwalls wants to merge 1 commit into
preetam/speculation-standardfrom
preetam/speculation-path-store
Open

feat(storage): speculation path set store#501
behinddwalls wants to merge 1 commit into
preetam/speculation-standardfrom
preetam/speculation-path-store

Conversation

@behinddwalls

@behinddwalls behinddwalls commented Aug 3, 2026

Copy link
Copy Markdown
Collaborator

Summary

Why?

The speculate run reads every head's path set at the start of a run and persists what it funds at the end. entity.SpeculationPathSet exists, but nothing can store one.

What?

Adds storage.SpeculationPathSetStore, keyed on the head batch ID, with a MySQL implementation, schema, mocks, and contract tests.

The mutation is a conditional whole-item put — Update replaces the set, guarded on version — because a conditional put on a key is the primitive every backend offers directly; a field-level update is the one shape non-SQL backends would have to emulate with a read-modify-write. Version arguments are explicit and the entity's own Version field is ignored, per the storage README.

No secondary index: callers that want a queue's live sets enumerate the heads from the batch listing they already hold and read each set by key.

Test Plan

bazel test //submitqueue/extension/storage/... — sqlmock coverage of get/create/update and every error contract, plus a pin that Update ignores the entity's Version field.

bazel test //test/integration/submitqueue/extension/storage/mysql:go_default_test — against real MySQL: round-trip (each stored ID still hashes its stored path), missing head, duplicate create, and a compare-and-swap race where the loser must not restore the path the winner dropped.

make fmt, make gazelle, make mocks

Stack

  1. docs(rfc): explain best-first speculation generation #513
  2. feat(speculation): generator contract and bestfirst impl #446
  3. feat(speculation): allocator contract and sticky impl #450
  4. feat(speculation): standard composed speculator #451
  5. @ feat(storage): speculation path set store #501
  6. feat(storage): path-build link store for per-path builds #502
  7. feat(orchestrator): dispatch builds per speculation path #503
  8. feat(orchestrator): poll builds and stop the ones nothing wants #504
  9. fix(orchestrator): mint distinct message IDs for cancel re-publishes #505
  10. feat(orchestrator): re-plan the queue from the Speculator each run #506
  11. feat(orchestrator)!: finalize batches from their speculation paths #507
  12. feat(orchestrator): compose per-queue speculators and turn speculation on #508

@behinddwalls
behinddwalls marked this pull request as ready for review August 3, 2026 16:46
@behinddwalls
behinddwalls requested review from a team and sbalabanov as code owners August 3, 2026 16:46
@behinddwalls
behinddwalls force-pushed the preetam/speculation-path-store branch 2 times, most recently from c56ac02 to e481adf Compare August 4, 2026 01:58
@behinddwalls
behinddwalls force-pushed the preetam/speculation-path-store branch from e481adf to c4f175e Compare August 4, 2026 05:11
@behinddwalls
behinddwalls force-pushed the preetam/speculation-path-store branch from c4f175e to 5483221 Compare August 5, 2026 03:15
@behinddwalls
behinddwalls force-pushed the preetam/speculation-path-store branch from 5483221 to cdce63d Compare August 5, 2026 03:26
@behinddwalls
behinddwalls force-pushed the preetam/speculation-path-store branch from cdce63d to 6989aee Compare August 5, 2026 03:37
@behinddwalls
behinddwalls force-pushed the preetam/speculation-path-store branch from 6989aee to 3f6c278 Compare August 5, 2026 03:43
@behinddwalls
behinddwalls force-pushed the preetam/speculation-path-store branch from 3f6c278 to f9aa731 Compare August 5, 2026 04:11
## Summary

### Why?

The speculate run reads every head's path set at the start of a run and persists what it funds at the end. `entity.SpeculationPathSet` exists, but nothing can store one.

### What?

Adds `storage.SpeculationPathSetStore`, keyed on the head batch ID, with a MySQL implementation, schema, mocks, and contract tests.

The mutation is a conditional whole-item put — `Update` replaces the set, guarded on version — because a conditional put on a key is the primitive every backend offers directly; a field-level update is the one shape non-SQL backends would have to emulate with a read-modify-write. Version arguments are explicit and the entity's own `Version` field is ignored, per the storage README.

No secondary index: callers that want a queue's live sets enumerate the heads from the batch listing they already hold and read each set by key.

## Test Plan

✅ `bazel test //submitqueue/extension/storage/...` — sqlmock coverage of get/create/update and every error contract, plus a pin that `Update` ignores the entity's `Version` field.

✅ `bazel test //test/integration/submitqueue/extension/storage/mysql:go_default_test` — against real MySQL: round-trip (each stored ID still hashes its stored path), missing head, duplicate create, and a compare-and-swap race where the loser must not restore the path the winner dropped.

✅ `make fmt`, `make gazelle`, `make mocks`
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant