RealSpill v1.0: a benchmark for causal effect estimation under network interference built from natural experiments in science

RealSpill is a benchmark for estimating direct and spillover effects under network interference. It turns three dated interventions recorded in open scholarly data into causal-inference tasks on real graphs: R, retraction (260,370 papers): papers retracted in 2012–2021 (Retraction Watch); papers are linked by shared authors and by pre-event citations; outcome log(1 + citations in years T+1 to T+3). M, international mobility (192,058 authors): first international moves in 2010–2018; authors are linked by coauthorship in T−3 to T−1 and by shared institution; outcome log(1 + papers in T+1 to T+3). D, journal delisting (11,069 journals): journals whose Scopus coverage ended in 2014–2021; journals are linked by author overlap and by publisher; outcome log(1 + articles in T+2 to T+3). Every task is scored against three tiers of truth: NULL (pseudo-treatments on units that no real intervention reached, so both true effects are exactly zero), PLASMODE (exactly known effects injected into the real outcomes, with seven stress scenarios: heterogeneity, dose response, wrong interference relation, 30% missing edges, 30% spurious edges, hidden confounding) and REAL (the real interventions, scored against reference intervals from quasi-experimental designs that pass a pre-trend equivalence test). The archive RealSpill-v1.0.zip contains the three family tables (Parquet), the design-based reference effects, the realspill Python package (loaders, tier generators, 16 estimators, design-based references), all scripts that build the tables and reproduce the experiments, all 5,248 benchmark runs and the aggregated result tables, figure source data, a datasheet, a data dictionary, a leaderboard, Croissant metadata and SHA-256 checksums. Author identifiers in family M are pseudonymized. How to download. Because of upload limits, the archive RealSpill-v1.0.zip (137,591,338 bytes) is stored in nine pieces, RealSpill-v1.0.zip.001 to RealSpill-v1.0.zip.009. Download all nine pieces into one folder and join them in order: cat RealSpill-v1.0.zip.0* > RealSpill-v1.0.zip on Linux or macOS, or copy /b RealSpill-v1.0.zip.001+RealSpill-v1.0.zip.002+RealSpill-v1.0.zip.003+RealSpill-v1.0.zip.004+RealSpill-v1.0.zip.005+RealSpill-v1.0.zip.006+RealSpill-v1.0.zip.007+RealSpill-v1.0.zip.008+RealSpill-v1.0.zip.009 RealSpill-v1.0.zip on Windows. Then verify the result against RealSpill-v1.0.zip.sha256 (shasum -a 256 -c RealSpill-v1.0.zip.sha256) and unzip it. See also HOW_TO_JOIN.txt. Data (data/, references/, results/, figures/) are licensed under CC BY 4.0; code (realspill/, scripts/) under the MIT License. The data are derived from OpenAlex (CC0), the Retraction Watch database distributed by Crossref, and the Scopus list of discontinued titles. Please also cite OpenAlex and the Retraction Watch database when using the data. Companion paper: H. Liu, X. Zong, J. Chen and J. Xiong, “RealSpill: Benchmarking Causal Effect Estimation under Network Interference with Natural Experiments in Science,” submitted to IEEE Transactions on Knowledge and Data Engineering.