Skip to content

Commit fd8e42d

Browse files
committed
README: add --seed to global flags, fix benchmark default, cleanup examples
1 parent 0481b47 commit fd8e42d

1 file changed

Lines changed: 10 additions & 6 deletions

File tree

README.md

Lines changed: 10 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,7 @@ profine run-all examples/minGPT/projects/chargpt/chargpt.py \
3030
--hardware 1x_a100 --steps 25 --warmup 10 --seed 42
3131
```
3232

33-
Full artifacts (JSON + Markdown reports for every pipeline step) live in [`examples/minGPT/profine_output/`](examples/minGPT/profine_output/) — read [`SUMMARY.md`](examples/minGPT/profine_output/SUMMARY.md) first. As shown in the [demo](https://youtu.be/CY9aW1Dcrn0).
33+
Full artifacts (JSON + Markdown reports for every pipeline step) live in [`examples/minGPT/profine_output/`](examples/minGPT/profine_output/). Read [`SUMMARY.md`](examples/minGPT/profine_output/SUMMARY.md) first.
3434

3535
## Install
3636

@@ -85,7 +85,7 @@ read → profile → interpret → suggest → edit → benchmark
8585

8686
Each step reads the previous step's output from `profine_output/`.
8787

88-
Global flags (all commands): `--provider {openai,anthropic,local}` (default `openai`), `--api-key`, `--model`, `--base-url` (for `local`), `-o/--output` (default `profine_output`), `--prefs`.
88+
Global flags (all commands): `--provider {openai,anthropic,local}` (default `openai`), `--api-key`, `--model`, `--base-url` (for `local`), `--seed` (best-effort, makes LLM rankings reproducible), `-o/--output` (default `profine_output`), `--prefs`.
8989

9090
### Auto (`run-all`)
9191

@@ -122,7 +122,7 @@ No additional flags. Output: `profine_output/read/architecture_record.json`
122122
Instrument the script and run on Modal with torch.profiler; collects step times, kernel breakdown, GPU utilization, and memory.
123123

124124
```bash
125-
profine profile nanoGPT/train.py --hardware 1x_a100 --steps 20 --warmup 10
125+
profine profile nanoGPT/train.py --hardware 1x_a100
126126
```
127127

128128
| Flag | Default | Description |
@@ -190,18 +190,22 @@ Output: `profine_output/edit/edited_train.py`, `profine_output/edit/files/`, `pr
190190
Run original and optimized back-to-back on the same hardware. Patched library files in `profine_output/edit/files/` are overlaid on the optimized run. Loss tolerance auto-widens for numerics-perturbing classes (BF16/mixed precision: rtol 5%, quantization: rtol 10%).
191191

192192
```bash
193-
profine benchmark nanoGPT/train.py --optimized profine_output/edit/edited_train.py --hardware 1x_a100 --steps 20 --warmup 10
193+
# Picks up the editor's most recent output automatically:
194+
profine benchmark nanoGPT/train.py --hardware 1x_a100
195+
196+
# Or point at a specific optimized script:
197+
profine benchmark nanoGPT/train.py --optimized profine_output/edit/edited_train.py
194198
```
195199

196200
| Flag | Default | Description |
197201
|---|---|---|
198-
| `--optimized` | required | Path to the optimized script |
202+
| `--optimized` | `<output>/edit/edited_train.py` | Path to the optimized script |
199203
| `--hardware` | `1x_a100` | Hardware preset name |
200204
| `--steps` | `60` | Total optimizer steps |
201205
| `--warmup` | `30` | Warmup steps |
202206
| `--rtol` | `0.01` | Relative tolerance for loss check (auto-widened) |
203207
| `--atol` | `0.0001` | Absolute tolerance for loss check (auto-widened) |
204-
| `--edit-dir` | `<output>/edit` | Directory whose `files/` subtree is overlaid |
208+
| `--edit-dir` | `<output>/edit` | Directory whose `files/` subtree is overlaid onto the optimized run (multi-file edits) |
205209
| `--timeout` | `900` | Modal container timeout (s) |
206210
| `--warmstart` | off | Reuse deployed Modal app between runs |
207211

0 commit comments

Comments
 (0)