Replication package: Would an AI reference recommender have found what information scientists cited? A temporally split benchmark across 45 years of Journal of Information Science
收藏资源简介:
Replication package for a study that tests whether AI reference recommenders recover the references that information scientists actually cited. The benchmark is built from every machine-readable reference list in Journal of Information Science (1979–2026), retrieved from the public OpenAlex API on 2026-11-03. Contents: benchmark construction and evaluation code (Python); experiment configuration, seeds and model settings; derived result files: per-query metrics, bootstrap intervals, paired tests, ablations, robustness checks, error analysis, six figures; data manifest with source, licence, processing and known coverage limitations; README with reproduction instructions and expected results. Scope: 2,315 citing articles, 54,073 candidate works and 71,473 positive citation pairs; 545 test articles from 2021 onward. Systems compared: BM25, TF-IDF, BGE-small, a bibliometric heuristic and reciprocal rank fusion. Licence: code MIT; documentation, derived aggregates and figures CC BY 4.0; source metadata CC0 1.0 (OpenAlex). Raw abstract text is not redistributed. Ethics: public bibliographic metadata only; no human participants or individual-level personal data.



