Chapter 67

TrainMode — 29 named updates

github.com/openfluke/welvet/layers/parallel✅ 29 modes

Why it exists

Backprop is one update, not the only one. Credit assignment (broadcast gap, head proxy, sparse duty clock) has to be a named axis you can race — not a comment in a notebook. Cameral Mix also needs one TrainMode per hemisphere on the same loss.

What it is

parallel.TrainMode: AllNamedTrainModes() = 29 (Inherit omitted). Stack-local Split/Alt plus Step* 1D pipe twins and Mesh* grid schedulers. TrainStackMSE / TrainStackCE honour BranchModes. Display names use Short() / ShortTrainMode. Rival metric is hard Acc vs StepBP; Lucy Score is Tput × Avail × Acc — do not mix those sentences.

Loss gap (MSE or CE)

MSE:  L = (1/d) ||ŷ − t||²     g_y = (2/d)(ŷ − t)
CE:   L = −mean log softmax(ŷ)_class     g_y = (p − t) / B

Head always sees g_y. Classification hosts call TrainStackCE so Acc can leave chance; MSE on a one-hot stays uniform. What each mode does with g_y is the whole story.

Families (29 named tokens)

AllCreditTrainModes() = 16 stack-local Split/Alt plus Step* credit twins (no Mesh). AllMeshCreditTrainModes() = 4 Mesh credit (no HeadProxy / Linear / LinearCache / HeadProxyAsync Mesh twins). AllStackLocalTrainModes() = 22 (no Inherit, no Mesh*). AllNamedTrainModes() = 29 — the Test49 / test50 set. IsLineStep() = 11 (StepBP, StepTween, StepTweenChain, StepTweenSplit, StepTweenAlt, plus the six Step* credit twins).

Tables and logs use Short() / ShortTrainMode (legend: [T]=Tween [S]=Split [FP]=FastProxy [L]=Linear [HP]=HeadProxy). Persistence and ParseTrainMode still use the full String() token. New Step* iota values were appended so old uint8 numbers stay stable.

Equations

Backprop (the rival)

head:   g_head = J_head^T g_y
hemi:   g_hemi = J_hemi^T g_head
stem:   g_stem = J_stem^T g_hemi
dW_i from local Backward(g_i, x_i)

SIMD GEMV. This is who FastProxy has to beat on hard Acc — not Lucy Score.

Tween — broadcast, half LR

g_i = P(g_y)     η ← η/2
dW_i = localBackward(g_i, x_i)

Blind to W^T sign. Sine Acc collapses. MeshTween is the volumetric scheduler of this family, not a secret FastProxy.

TweenSplit — even split

g_i = (1/N) P(g_y)

Still not J^T. Cheap Acc ceiling. Score can rise because the update is cheap (Avail), not because g is better.

TweenAlt — Split then Tween

Per sample, AltTimes times (default 1): Split from live g_y → re-forward → Tween from g_y′ (half LR). Extra forwards kill Avail → Score last.

HeadProxy

g_proxy = J_head^T g_y = W_head^T (g_y ⊙ act′(pre_head))
hidden i = 1…N−1:  g_i = 1/(N−1) P(g_proxy)
                   dW_i = g_i x_i^T     (no discarded W^T)

One real J_head^T. Hemispheres do not get J_hemi^T.

FastProxy

g_proxy = W_head^T g_y          ← skip act′
head dW still uses act′
hidden: same 1/(N−1) P(g_proxy) dW-only as HeadProxy

DFA with B := W_head^T, not a learned random B. On AAI test50 sine, FastProxy SoftAcc often sits above StepBP while both are at 100% hard Acc — that is the FastProxy vs BP sentence.

Linear

g_head↓  = g_y
g_hemi↓  = W_head^T g_y          (siblings share the vector)
g_stem↓  = Σ_hemi W_hemi^T g_hemi↓
then every leaf: g_i = (1/N) P(g_i↓),  dW_i = g_i x_i^T

Same O(N²) class as backprop. Score stays StepBP-class unless Acc is way up.

LinearCache — dead control

every 20 steps: full Linear walk, cache g_i↓
else: g_i ← g_i^cache · ||g_y||_live / ||g_y||_cache

Norm scaling cannot recover sign flips after a frequency switch. If this wins sine, the board is lying.

HeadProxyAsync

g_hidden^(T) = 1/(N−1) P(g_proxy^(T−1))
head computes g_proxy^(T) = W_head^T g_y^(T) for next step

First sample seeds live. Stale sign on XOR. Not EMA.

Sparse — duty clock

g_proxy = W_head^T g_y
dW_head = g_y x_head^T
k = t mod (N−1)
dW_k    = P(g_proxy) x_k^T
other leaves: dW = 0 this sample

Real FLOP cut → Avail 40–50% → Lucy Score explodes. That is a duty clock, not a smaller big-O than backprop and not a better chain rule. On test50 copy, Sparse often loses Acc vs StepBP while winning Score.

Step* — 1D systolic pipe

The Step prefix on a stack mode is a schedule, not a second leftover-forward pass and not Mesh*. IsLineStep → TrainLine / trainStackLine: one sample enters child 0 per tick; every in-flight sample advances one layer; the output (and the train event) is the sample that entered D ticks ago. Fill ticks do not update. Serve stays a full ForwardStack.

Same family update on the Sandwich whether Step or not — HeadProxy / Linear / FastProxy / Sparse / Async credit walk trainTweenSplitLeaves on both twins. Mesh* still requires a Grid (RequiresGrid). There is no Mesh HeadProxy / Linear / LinearCache / HeadProxyAsync.

Mesh*

MeshBP = volumetric training.Step. MeshTween = StepMesh. MeshTweenChain = StepTween. Mesh Split / Alt / FastProxy / Sparse credit the placed stack under a grid walk. On origin-only 1³/2³/3³ (rest IsDisabled) Mesh* usually matches the stack twin. Cube size is hop topology, not 27 sandwiches.

Where it is raced

• w2a Test49 — permutation smoke: 29 modes × 1³/2³/3³ × Parallel / Bicameral / poly kinds, origin-only. In [0] Run ALL. Not a Lucy race.
• AAI test48 — credit sweep (layers × dtypes × short jobs) with these equations.
• AAI test50 — FP32 Lucy race, all 29, cams 1–3, cubes 1–3. Copy: Split/Alt family +5 to +12 Acc vs StepBP. Sine: Acc ceiling; FastProxy SoftAcc is the knife. XOR: 4-point parking lot (75%). Sparse wins Score, not Acc.

Cameral why + sandwich stem→mid→head: §68. Measuring math: §66.

package main

import (
	"fmt"

	"github.com/openfluke/welvet/layers/parallel"
)

func main() {
	named := parallel.AllNamedTrainModes()
	line := 0
	for _, m := range named {
		if m.IsLineStep() {
			line++
		}
	}
	fp, err := parallel.ParseTrainMode("stepfastproxy")
	fmt.Println("named", len(named), "linestep", line)
	fmt.Println("stepfastproxy", fp.Short(), err)
	fmt.Println(parallel.ShortTrainModeLegend)
}

named 31 linestep 11
stepfastproxy Step[T][S][FP] <nil>
[T]=Tween  [S]=Split  [FP]=FastProxy  [L]=Linear  [HP]=HeadProxy  [F]=Freeze  [Sh]=Shadow  [A]=Adv  [M]=Memory