Chapter 11

layers/dense — MatVec microkernel

github.com/openfluke/welvet/layers/dense✅

Why it exists

Most FLOPs are W@x. One Dense stack owns FormatNone×34 and all quants × three backends so every composite proj shares one correctness surface.

What it is

New / NewConfigured[T], Forward/Backward (dispatch on Exec.Backend), Place, ApplyGradSGD. SIMD: fused Dot* for classic Q*, k/IQ (group scales), AffinePacked code-dot — no F32 inflate. Composites (MHA, SwiGLU, CNN im2col, RNN/LSTM) reuse Dense children.

Go example

examples/11-dense/main.go

Run:cd welvet/examples/11-dense && source ../env.sh && go run .

package main

import (
	"fmt"

	"github.com/openfluke/welvet/core"
	"github.com/openfluke/welvet/layers/dense"
	"github.com/openfluke/welvet/quant"
)

func main() {
	init := make([]float32, 4*8)
	l, err := dense.NewConfigured(8, 4, core.ActivationReLU, core.DTypeFloat32, quant.FormatNone, init)
	if err != nil {
		panic(err)
	}
	l.Exec.Backend = core.BackendCPUTiled
	x := core.NewTensor[float32](1, 8)
	pre, post, err := dense.Forward(l, x)
	if err != nil {
		panic(err)
	}
	gIn, gW, err := dense.Backward(l, post, x, pre)
	_ = dense.ApplyGradSGD(l, gW, 1e-3)
	fmt.Println(len(gIn.Data), len(gW.Data))
}

Output

exit 0 · last run via go run .

8 32