Webis-SR4ALL-26: A Large-Scale, Cross-Disciplinary Corpus of Systematic Reviews

Overview Webis-SR4ALL-26 is a large-scale corpus of 301,871 systematic reviews spanning 27 scientific disciplines. The corpus is derived from OpenAlex and enriched with structured methodological information extracted from full texts where available. The dataset is designed to support research on systematic review retrieval, screening, and meta-research across domains. It provides a unified, open infrastructure for large-scale experimentation under consistent indexing and citation conditions. This release accompanies the paper: TBD Files Included 1. sr4all_full.jsonl This file contains 301,871 systematic reviews in JSON Lines format (one JSON object per line). Each record includes: OpenAlex Metadata (available for all reviews) id (OpenAlex ID) title doi abstract (if available) year type source cited_by_count referenced_works_count referenced_works (OpenAlex IDs) pdf_url (if available in OpenAlex) language field subfield topics keywords authors All citation links are derived exclusively from OpenAlex indexed reference lists. No external reference parsing or enrichment was performed. Structured Methodological Fields (subset only) For reviews where full text was available and successfully parsed, additional fields may be present: objective research_questions keywords_used exact_boolean_queries inclusion_criteria exclusion_criteria n_studies_initial n_studies_final year_range year_range_normalized databases_used snowballing These fields are only present where full text was available and the extraction pipeline produced verified outputs. Missing fields are represented as null or absent. Full texts themselves are not included in this release due to copyright restrictions. Where available, pdf_url links provided by OpenAlex are included for research access. 2. sr4all_normalised_queries.jsonl This file contains normalized Boolean queries derived from reported search strategies. Each record includes: id (OpenAlex ID; maps directly to sr4all_full.jsonl) boolean_queries (list of normalized Boolean expressions) Normalization removes database-specific syntax and retains: Topical terms Logical operators: AND, OR, NOT Temporal constraints and metadata filters are not embedded in the Boolean string and must be applied externally. Only reviews with extractable and normalizable search strategies are included in this file. Queries that could not be normalized are omitted. Corpus Construction Summary Initial candidate retrieval from OpenAlex using title-based heuristics. Deduplication via DOI, OpenAlex ID, and normalized titles. Metadata filtering (English language, DOI present, non-update reviews). Integration of benchmark systematic review datasets. Final corpus size: 301,871 systematic reviews. Full-text PDFs successfully retrieved and parsed: 72,678. Structured methodological extraction applied to parsed full texts. Extraction follows a verify-then-repair pipeline prioritizing precision over recall. Format Both files use JSON Lines (.jsonl) format: UTF-8 encoding One JSON object per line No trailing commas Suitable for streaming and large-scale processing. What This Corpus Enables The corpus supports multiple lines of research: 1. Retrieval Evaluation Execute normalized Boolean queries against OpenAlex. Compare retrieved results against cited reference sets. Compute precision, recall, and F-scores. Analyze cross-domain retrieval behavior. 2. Screening and Ranking Use reference sets as weak supervision for relevance modeling. Benchmark citation screening methods at scale. Study cross-domain generalization of screening models. 3. Query Generation and Reformulation Learn query structure from extracted expert-written strategies. Compare LLM-generated queries against author-reported strategies. Analyze logical structure across disciplines. 4. Meta-Research and Science-of-Science Study reporting practices (objectives, inclusion criteria, databases). Analyze disciplinary differences in search strategy formulation. Examine temporal trends in systematic review methodology. 5. Large-Scale Observational Analysis Characterize differences in methodological rigor across fields. Study variation in Boolean complexity and vocabulary. Analyze inclusion criteria formulations across domains. Important Limitations Full texts are not distributed due to copyright restrictions. Structured fields are only available for reviews with accessible and successfully parsed PDFs. Extraction prioritizes precision; some valid information may remain unstructured. Reference sets reflect cited works, not strictly included-study gold standards. Normalized queries are simplified approximations of original database-specific queries. Intended Audience Information retrieval researchers Systematic review automation researchers Bibliometrics and science-of-science researchers LLM researchers working on grounded extraction and query generation Developers of evidence synthesis toolsOverview Webis-SR4ALL-26是一个大规模语料库，包含301,871篇系统综述，覆盖27个科学学科。该语料库源自OpenAlex(OpenAlex)，并针对可获取全文的综述补充了从全文中提取的结构化方法学信息。 本数据集旨在支持跨领域的系统综述检索、筛选与元研究相关研究，可为一致索引与引用条件下的大规模实验提供统一的开放基础设施。 本数据集发布配套论文信息暂未公布（TBD）。 Files Included 1. sr4all_full.jsonl 该文件以JSON Lines格式存储301,871篇系统综述（每行对应一个JSON对象）。 每条记录包含以下内容： OpenAlex元数据（所有综述均包含该部分） - 标识符（OpenAlex标识符） - 标题 - DOI - 摘要（若可获取） - 发表年份 - 文献类型 - 来源出版物 - 被引频次 - 参考文献数量 - 参考文献（OpenAlex标识符列表） - PDF链接（若OpenAlex中可获取） - 语言 - 学科领域 - 子学科领域 - 主题 - 关键词 - 作者 所有引用链接均仅源自OpenAlex索引的参考文献列表，未进行外部参考文献解析或补充处理。 结构化方法学字段（仅部分综述包含） 对于可获取全文且成功解析的综述，可能包含以下额外字段： - 研究目标 - 研究问题 - 所用关键词 - 精确布尔查询式 - 纳入标准 - 排除标准 - 初始纳入研究数量 - 最终纳入研究数量 - 年份范围 - 标准化年份范围 - 所用数据库 - 滚雪球检索 仅当可获取全文且提取流程生成了验证后的结果时，才会包含这些字段。缺失的字段将以null或直接省略表示。 由于版权限制，本发布版本不包含全文本身。若可获取，将附带OpenAlex提供的PDF链接以供研究使用。 2. sr4all_normalised_queries.jsonl 该文件包含从已报告的检索策略中提取的标准化布尔查询式。 每条记录包含以下内容： - 标识符（OpenAlex标识符，与sr4all_full.jsonl中的id一一对应） - 布尔查询式（标准化布尔表达式列表） 标准化处理会移除数据库专属语法，保留以下内容： - 主题术语 - 逻辑运算符：AND（与）、OR（或）、NOT（非） 时间约束与元数据筛选器并未嵌入布尔字符串中，需外部另行应用。 本文件仅包含可提取且可标准化的检索策略对应的综述，无法标准化的查询式将被省略。 Corpus Construction Summary 语料库构建流程总结： - 基于标题启发式规则从OpenAlex中检索初始候选综述 - 通过DOI、OpenAlex标识符与标准化标题进行去重 - 元数据筛选（仅保留英文文献、带有DOI的非更新型综述） - 整合基准系统综述语料库 - 最终语料库规模：301,871篇系统综述 - 成功获取并解析的全文PDF数量：72,678篇 - 对已解析的全文应用结构化方法学信息提取 - 提取流程遵循"先验证后修复"的管线，优先保证准确率而非召回率 Format 格式说明： 两个文件均采用JSON Lines（.jsonl）格式： - UTF-8编码 - 每行一个JSON对象 - 无尾部逗号 - 适用于流式处理与大规模数据分析 What This Corpus Enables 本语料库支持以下多类研究方向： 1. 检索评估 - 针对OpenAlex执行标准化布尔查询式 - 将检索结果与参考文献集进行比对 - 计算准确率、召回率与F1值 - 分析跨领域检索行为 2. 筛选与排序 - 以参考文献集作为弱监督信号进行相关性建模 - 大规模基准测试引用筛选方法 - 研究筛选模型的跨领域泛化能力 3. 查询式生成与重构 - 从已提取的专家撰写的检索策略中学习查询式结构 - 对比大语言模型（Large Language Model, LLM）生成的查询式与作者报告的检索策略 - 分析不同学科间的逻辑结构差异 4. 元研究与科学学研究 - 研究报告撰写规范（如研究目标、纳入标准、所用数据库等） - 分析不同学科在检索策略制定上的差异 - 探究系统综述方法学的时间演变趋势 5. 大规模观测分析 - 刻画不同学科间方法学严谨性的差异 - 研究布尔查询式复杂度与词汇使用的差异 - 分析不同领域的纳入标准表述方式 Important Limitations 重要局限性： - 由于版权限制，本发布版本不包含全文内容 - 结构化字段仅对可获取且成功解析的PDF的综述可用 - 提取流程优先保证准确率，部分有效信息可能未被结构化处理 - 参考文献集仅反映被引文献，并非严格意义上的纳入研究金标准 - 标准化查询式是对原始数据库专属查询式的简化近似 Intended Audience 目标受众： - 信息检索研究者 - 系统综述自动化研究者 - 文献计量学与科学学研究者 - 开展基于真实文本的信息抽取与查询式生成研究的大语言模型研究者 - 证据合成工具开发者