Geoffrey-Wang/IntegriRef-Bench

--- license: mit task_categories: - text-classification - fact-checking language: - en tags: - citation-verification - hallucination-detection - scientific-integrity - bayesian-scoring - reference-checking size_categories: - 10K<n<100K --- # IntegriRef-Bench Benchmark dataset for multi-level reference integrity verification, accompanying the IntegriRef framework. ## Dataset Splits | Split | Rows | Description | |-------|------|-------------| | `reference_verification` | 1,926 | Golden benchmark + crawled verification cases (hallucinated, real, chimera, retracted) | | `signal_unit_tests` | 403 | Per-signal unit tests for 14 Bayesian signal types | | `l1_intent` | 20 | Citation intent classification test pairs | | `l2_nli` | 20 | Claim-evidence NLI alignment test pairs | | `graph_anomaly` | 3,030 | Citation graph anomaly cases (rings, temporal, orphan clusters) | | `retracted_papers` | 6,391 | Retracted papers from Crossref + PubMed with real controls | | **Total** | **11,790** | | ## Usage ```python from datasets import load_dataset ds = load_dataset("Geoffrey-Wang/IntegriRef-Bench") # Or load a specific split ref_ver = load_dataset("Geoffrey-Wang/IntegriRef-Bench", data_files="reference_verification.jsonl") ``` ## Sources - **Retracted papers**: Crossref `update-to` retraction markers + PubMed retraction notices - **Hallucinated references**: Programmatically generated with verified non-existence - **Chimera references**: Real DOIs paired with swapped metadata - **Graph anomaly cases**: Documented citation cartels (Brazilian 2009-2013, Ji-Huan He, IOP 2024) - **Temporal anomalies**: OpenAlex citation graph analysis - **Statistical cases**: GRIM test + statcheck from PMC Open Access full texts ## Citation ```bibtex @inproceedings{integriref2026, title={IntegriRef: A Five-Layer Bayesian Framework for Cross-Domain Reference Integrity Verification}, author={Anonymous}, booktitle={KnowFM Workshop at ACL}, year={2026} } ``` ## License MIT