遇见数据集

Data and code for "Outlier-channel identity is a seed-broken symmetry deposited early in pretraining"

收藏
Zenodo2026-07-20 更新2026-08-02 收录
官方服务:

资源简介:

Analysis code and complete result files backing every number in the paper"Outlier-channel identity is a seed-broken symmetry deposited early in pretraining". A ~0.1% minority of hidden dimensions dominates transformer activation magnitude. That suchoutlier channels emerge early, track token frequency in magnitude, and can be suppressed isestablished; what no prior account names is WHICH index a given run settles on. This archivecontains the evidence that the index is a seed-broken symmetry, deposited in the first fewpercent of training. Contents (78 files):- prereg/ : four pre-registration documents (the blind-prediction commitments, frozen before any final checkpoint was inspected)- results/ : 24 verdict and analysis JSONs, including the blind-recovery verdict (6/7 lineage families, 15/16 runs), the within-model decoy null (19/22, p=4.7e-6), the content-controlled 3x2 causal pretraining grids at 124M and 1B, the initialisation mechanism probe, the selection-versus-amplification extraction, quantisation-survival tables, and the pythia-12b extension trial with its full audit trail (frozen prediction, early/final profiles, SHA-256 integrity audit)- scripts/ : 11 analysis and experiment scripts- README.md : a claim-to-file map linking each number in the paper to the file that produced it No model weights are redistributed. All analysed models are public Hugging Face checkpoints;each result file records the exact repository and revision it was produced from, and embeds theSHA-256 of every weight blob it read, so the integrity audit is reproducible from this archivealone. Code: MIT. Result files: CC BY 4.0.

提供机构:
Zenodo
创建时间:
2026-07-20
二维码
社区交流群
二维码
科研交流群
商业服务