Skip to content

fix: use CPU-aware prover scheduler - #897

Open
jotabulacios wants to merge 4 commits into
mainfrom
fix/cpu-prover-scheduler
Open

fix: use CPU-aware prover scheduler#897
jotabulacios wants to merge 4 commits into
mainfrom
fix/cpu-prover-scheduler

Conversation

@jotabulacios

Copy link
Copy Markdown
Collaborator

This PR fixes the CPU proving regression introduced by the table scheduler.

The GPU scheduler uses external driver threads to admit and overlap table work. On CPU, that caused table tasks to launch nested Rayon work from outside the Rayon pool, leading to oversubscription, cache contention, and memory-bandwidth contention.

The CPU path now:

  • schedules table work through the existing Rayon pool;
  • limits table concurrency in bounded chunks; and
  • separates auxiliary-trace construction/commitment from rounds 2–4 with a CPU phase barrier.

The CUDA path and proof format are unchanged.

@jotabulacios

Copy link
Copy Markdown
Collaborator Author

/bench

@github-actions

github-actions Bot commented Aug 4, 2026

Copy link
Copy Markdown

Benchmark — real block (ethrex_mainnet_25368371.bin) (median of 3)

continuations · epoch 2^22 · 11 epochs

Metric main PR Δ
Peak heap 47250 MB 52967 MB +5717 MB (+12.1%) 🔴
Prove time 141.389s 124.107s -17.282s (-12.2%) 🟢

🎉 Improvement on the real block — prove time down 12.2%.

Prove-time spread 0.7% (123.672s / 124.107s / 124.480s)

Memory Growth

ethrex distinct-account transfers · default parallelism · 1 sample per point

Transfers main (MB) PR (MB) Δ
4 14086 15070 +984 MB (+7.0%)
8 16094 19692 +3598 MB (+22.4%)
12 19602 24761 +5159 MB (+26.3%)
16 24994 30187 +5193 MB (+20.8%)
20 26996 32290 +5294 MB (+19.6%)

Growth rate: 1123 MB / transfer (main: 868, Δ: +29.4%)
Fit: R² = 0.9845 (main: 0.9749)

⚠️ Memory scaling regression — growth rate increased by +29.4%

Commit: b5256ee · Baseline: cached · Runner: self-hosted bench

@jotabulacios
jotabulacios marked this pull request as ready for review August 4, 2026 20:33
@diegokingston

Copy link
Copy Markdown
Collaborator

/bench

@MauroToscano

Copy link
Copy Markdown
Contributor

/bench-growth

@MauroToscano MauroToscano left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Feedback incoming

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants