Add MNIST Muon microbatch RG spectrum experiment

[charlesmartin14]

What changed

• adds an opt-in MNIST MLP3 runner that reuses the baseline model, deterministic MNIST split, Muon-with-auxiliary-AdamW optimizer, gradient clipping, and update-level warmup/cosine schedule
• saves fc1.weight, fc2.weight, and fc3.weight after each selected optimizer microbatch, including the initialized step-zero state and an append-safe checkpoint index
• adds storage estimation and an explicit guard for full 30-epoch every-step captures, which are roughly 32 GiB in float32
• adds spectrum utilities for ordinary weight ESDs, supported relative-flow maps between successive checkpoints, and abs(log(lambda)) deviations with the identity mode removed
• adds a notebook that fits each layer/spectrum with powerlaw.Fit and plots alpha versus optimizer step with an alpha=2 reference line
• explicitly expands the powerlaw alpha range from its version-2 default upper bound of 3 to [1.01, 10], and records tail size, tail fraction, KS distance, and boundary hits
• adds unittest-discoverable round-trip, algebra, synthetic-MNIST runner, and notebook-contract tests

Run

cd baseline
python -m pip install -e '.[experiment]'
rg-mnist-muon-microbatch \
  --data-dir ./data \
  --output-dir ./results/mnist_mlp3_muon_microbatch_500 \
  --max-steps 500 \
  --capture-every 1 \
  --overwrite

Then open baseline/notebooks/MNIST_MLP3_Muon_Microbatch_RG_ESD.ipynb and point RUN_DIR at the generated run.

Scientific scope

The notebook reports three candidates separately:

1. sigma(W_t)^2
2. sigma(J_t)^2, where the smaller supported square relative map is built with a pseudoinverse
3. abs(log(sigma(J_t)^2)), which removes the trivial identity/orthogonal mode at one

The relative-flow construction remains basis dependent; this PR implements the proposed numerical test rather than claiming a basis-invariant quotient.

Real-MNIST pilot

A separate Actions run trained the exact baseline on real MNIST for 500 optimizer microbatches and saved initialization plus all 500 updates. At step 500 it reached validation accuracy 97.36% and test accuracy 97.05%.

After refitting the saved spectra with 1.01 <= alpha <= 10, the strongest early signal is the middle hidden layer's log-flow-deviation spectrum:

• fc2.weight: late-step median alpha 1.9519, mean 1.9544, standard deviation 0.0177
• median fitted tail 165.5 / 512 modes (32.3%)
• median KS distance 0.0204

The other candidates do not show a comparable universal alpha-two result in the first 500 steps:

• ordinary weight ESDs are steep (alpha roughly 4.3, 6.2, and 7.6 by layer)
• raw relative-flow fits for fc1 and fc3 hit the expanded upper bound near 10 with poor KS distances, so those spectra are effectively identity-like rather than credible power laws
• fc2 raw relative flow has median alpha about 2.56 but a tiny fitted tail, roughly 7.5 of 512 modes
• fc3 has only ten singular values, so its fits are qualitative

This is one seed and only 500 of approximately 12,900 full-baseline steps. The fc2 log-flow result is therefore an encouraging layer-specific pilot, not yet evidence of a universal fixed point.

Validation

Passed on the current feature head:

• baseline source-syntax workflow
• complete core-baselines job, including all baseline unit tests
• synthetic MNIST-shaped smoke run using the real MLP3 and MuonWithAuxAdamW; initialization plus two successive optimizer-step checkpoints were written and reloaded
• relative-flow algebra and checkpoint round-trip tests
• notebook JSON/code-cell parsing and end-to-end smoke execution over three successive checkpoints
• pinned nanochat model/optimizer CPU preflight
• real-MNIST 500-step training, 501 checkpoint writes, spectrum construction, expanded-range power-law refit, plot generation, and artifact upload

The separate one-head-nanogpt job is red because of regressions already present on main: its reference configuration requires a missing muon_clip profile, and two existing angular notebooks no longer satisfy their contract tests. None of those failures involve files changed by this PR.

[charlesmartin14]

Rectangular FC1/FC2 follow-up completed on real MNIST (seed 1337, 500 optimizer microbatches, each sampled step compared with the immediately preceding checkpoint).

Implemented a gauge-aligned decomposition for full-rank rectangular matrices:

• W_t = B_t V_t^T for wide matrices
• Procrustes alignment of successive row-space bases
• core spectrum abs(log(sigma(B_t_aligned B_{t-1}^{-1})^2))
• separate Grassmann spectrum from squared principal angles theta_i^2
• exact reduction to W_t W_{t-1}^{-1} for square full-rank FC2

Late-step results (last ten sampled points, steps 410–500):

spectrum	layer	median alpha	std	median tail	median KS
core log deviation	FC2	1.9519	0.0177	165.5 / 512	0.0204
core log deviation	FC1	5.6584	1.6238	62.5 / 512	0.1126
angular theta^2	FC1	3.3705	1.3570	37 / 272	0.1131

At step 500, FC1 angular happened to fit alpha 2.1838 with 93 / 272 tail modes, but the trajectory is highly variable and does not provide a stable alpha-two signal. FC2 remains the only clean result: broad tail, low KS distance, and a stable exponent close to 2.

Conclusion: separating the rectangular core and Grassmann sectors fixes the mathematical incompleteness of the pseudoinverse operator, but it does not rescue FC1 in this 500-step pilot.