Trained checkpoints: spiking vs. transformer character-level language models on TinyStories
收藏资源简介:
Model weights for a comparison of spiking and conventional character-level language models on TinyStories: 533 PyTorch .pt files in eight archives. Each archive unpacks into the runs/ layout of the code repository and can be used on its own. spiking-lm-checkpoints-final-states.zip: 43 final model states, one per protocol run spiking-lm-checkpoints-gate-states.zip: 22 gate-stage states (memorization check and screening) spiking-lm-checkpoints-grid-T2-R2.zip: 90 intermediate checkpoints, spiking arms T2 and R2 spiking-lm-checkpoints-grid-GPT-PM-and-Q.zip: 90 intermediate checkpoints, GPT-PM and GPT-PM-Q spiking-lm-checkpoints-grid-LSTM.zip: 60 intermediate checkpoints, LSTM-PM and LSTM-12 spiking-lm-checkpoints-grid-GPT-FM-s10-s11.zip: 12 intermediate checkpoints, GPT-FM seeds 10 and 11 spiking-lm-checkpoints-grid-GPT-FM-s12.zip: 6 intermediate checkpoints, GPT-FM seed 12 spiking-lm-checkpoints-posthoc.zip (new in version 1.1): 210 states from 30 post hoc runs, 7 per run (6 intermediate checkpoints and the final state): LSTM-OM (LSTMOM, 10 runs), LSTM-T (LSTMUT, 5), A2 (A2, 5) and GPT-OM (GPTOM, 10), with SHA-256 checksums The first seven archives hold the seven planned arms and are unchanged from version 1.0. Code, per-run results and instructions: github.com/slastine/spiking-lm-project.



