遇见数据集

Reproduction package for "Question difficulty across models predicts factual errors of language models better than a single model detector"

收藏
Zenodo2026-11-01 更新2026-11-05 收录
官方服务:

资源简介:

Stored inputs, analysis scripts, reference results and frozen preregistrations for the article "Question difficulty across models predicts factual errors of language models better than a single model detector". The package covers a main panel of 17 open-weight instruction-tuned models, three replication corpora (Natural Questions, BioASQ, TriviaQA; 18 models), a nine-model design without the hard-question filter, the stored greedy and sampled answers, the question packs with reference aliases, the responses with the answer tokens marked in the extraction pipeline, and the GPU extraction scripts. run_all.sh recomputes 11 result files and compares them with the reference results; run_supplementary.py reproduces the supplementary checks. See README.md. Raw activation tensors (8.1 TB) are not included; they can be regenerated with the extraction scripts and the public model weights referenced in the data files.

提供机构:
Zenodo
创建时间:
2026-11-01
二维码
社区交流群
二维码
科研交流群
商业服务