A Tool in the Forward Pass: Frozen Tape Modules as Layers of a Small Language Model. Companion code, configurations, results and frozen modules

Companion material for the technical report A Tool in the Forward Pass: Frozen Tape Modules as Layers of a Small Language Model. A small neural module, trained alone on symbolic tapes and frozen, runs inside the forward pass of gemma-3-1b-it: a learned writer turns hidden states into a tape, the module reduces it to a result, and a learned reader returns the result to the model. Code (scripts/): the experiments, and the generator that writes every number and table of the paper from the results. Configurations (configs/): one per experiment, with the predictions registered before each run (the paper's Appendix B). Results (data/sweeps/): the measured results the paper is generated from. Frozen modules (checkpoints/): the arithmetic and the state-tracking tape modules (64k parameters each, four seeds). Paper (paper/): LaTeX source and PDF. See README.md in the archive for reproduction. Verify with SHA256SUMS. Licences: code MIT; data, results, checkpoints and the paper CC BY 4.0. Gemma weights are not included.