The VRAM-pressure fixes branch (#914) recovers from transient device declines at two sites — the R1 resident-aux commit and the R2 composition — by downloading the resident data and continuing host-backed. The same transient pressure also strikes downstream, where the fallbacks still hard-abort:
- R4 DEEP composition asserts (
crypto/stark/src/prover.rs ~2352 and ~2367)
- R2 composition-tree CPU-fallback asserts (~1797, ~1812 — these gate on part-evals emptiness, not
host_trace_empty())
- R3 barycentric per-buffer asserts (
crypto/stark/src/trace.rs ~809/~868)
Measured on rented RTX 5090s (real block 25368371 at epoch 2^22, plus synthetic pressure runs): the cliff sites dominate pressured failures — census across runs: R4 DEEP ×97, comp-tree ×42, R3 ×3; at ~7.6 GB free VRAM roughly 60% of proves died on a cliff. The covered recoveries were observed working (four downgrades in a single real-block prove) and the prove then died one round later on an uncovered site.
Fix shape: the R4 helpers take &LDETraceTable, while materialize_lde_trace_host needs &mut — either thread mutability through R4 or hoist a materialize to round entry when the device path declines. A deterministic test can use the test-cuda-faults hook pattern (needs a sticky mode: the drain-and-retry absorbs one-shot faults).
Full data: the PR #914 review thread.
The VRAM-pressure fixes branch (#914) recovers from transient device declines at two sites — the R1 resident-aux commit and the R2 composition — by downloading the resident data and continuing host-backed. The same transient pressure also strikes downstream, where the fallbacks still hard-abort:
crypto/stark/src/prover.rs~2352 and ~2367)host_trace_empty())crypto/stark/src/trace.rs~809/~868)Measured on rented RTX 5090s (real block 25368371 at epoch 2^22, plus synthetic pressure runs): the cliff sites dominate pressured failures — census across runs: R4 DEEP ×97, comp-tree ×42, R3 ×3; at ~7.6 GB free VRAM roughly 60% of proves died on a cliff. The covered recoveries were observed working (four downgrades in a single real-block prove) and the prove then died one round later on an uncovered site.
Fix shape: the R4 helpers take
&LDETraceTable, whilematerialize_lde_trace_hostneeds&mut— either thread mutability through R4 or hoist a materialize to round entry when the device path declines. A deterministic test can use thetest-cuda-faultshook pattern (needs a sticky mode: the drain-and-retry absorbs one-shot faults).Full data: the PR #914 review thread.