Systematic Reviews in Computer Science 2025

SR4CS: Systematic Reviews for Computer Science SR4CS is a benchmark dataset linking computer science systematic reviews (SRs) to their reported Boolean queries and curated reference pools, enabling reproducible research on Boolean query generation, retrieval effectiveness, and screening beyond the biomedical domain. Total SRs: 1,212 Resolved references: 104,316 (incl. 89,447 with abstracts) Tasks supported: Boolean query generation, retrieval benchmarking, screening Paper: tba Code: https://anonymous.4open.science/r/scolia26-sr4cs-2525 Contents 1) sr4cs.json Primary dataset of SR entries. A list of SR objects: { "id": "1000688", "databases": ["Google Scholar", "Scopus", "Web of Science"], "search_strings_boolean": [ "INTITLE (\"green\" OR \"sustainab*\") AND INTITLE (\"AI\" OR \"ML\" OR \"artificial intelligence\" OR \"machine learning\" OR \"deep learning\")" ], "year_range": "unbounded", "language_restrictions": ["English"], "inclusion_criteria": ["..."], "exclusion_criteria": ["..."], "topic": "Green AI", "objective": "Analyze Green AI literature ...", "research_questions": ["What are the characteristics ...?"], "snowballing": true, "sr_title": "A systematic review of Green AI.", "sr_doi": "10.1002/widm.1507", "sr_pdf_link": "https://...", "ref_id": [86, 113, 114, "..."], "num_refs": 97 } Field summary id (string): SR identifier (internal). databases (list[str]): Databases reported in the SR. search_strings_boolean (list[str]): Reported Boolean queries (verbatim). year_range (string): Year bounds, e.g., "2018-2022" or "unbounded". language_restrictions (list[str]): Reported language constraints. inclusion_criteria / exclusion_criteria (list[str]): As stated in the SR. topic, objective (string): SR topic & objective (verbatim). research_questions (list[str]): RQs (verbatim). snowballing (bool): Whether the SR reports snowballing/citation chasing. sr_title, sr_doi, sr_pdf_link (metadata for the SR). ref_id (list[int]): Integer IDs linking to the reference pool. num_refs (int): Count of linked references for this SR. 2) sr4cs_with_sql.json Same structure as sr4cs.json, plus translated SQL-style queries for uniform execution: sqlite_refined_queries (list[str]): SQLite FTS5 MATCH clauses translated from the reported Boolean strings (field-scoped to title/abstract). 3) Reference pool (metadata + abstracts) The full reference pool is provided in three complementary formats: refs.db — SQLite database with FTS5 indexes (ready for querying) refs.ndjson — Elasticsearch bulk dump (for index import) refs.parquet — Tabular format (not indexed) Reference fields (columns/keys): ref_id, arxiv, author, citation-number, collection-title, container-title, date, director, doi, edition, editor, genre, id, isbn, issue, location, note, pages, pmcid, pmid, producer, publisher, raw, source, title, translator, type, url, volume, title_norm, abstract Notes: ref_id is the primary key linking SR entries to references. title_norm is a normalized version of the title (lowercased, standardized spacing/characters) to support matching. title_norm and abstract are prioritized fields for retrieval. Many entries also include doi and url. Abstract coverage is ~89k; the remainder provide title-only metadata. Intended Use Boolean query generation: Compare generated queries against transleted expert-reported queries and their retrieval on the shared reference pool. Retrieval benchmarking: Evaluate title/abstract retrieval engines (e.g., SQLite FTS5) on realistic CS SR topics. Screening experiments: Use the linked reference pools to prototype screening/prioritization pipelines. Provenance & Construction (summary) Candidate SRs collected from DBLP by title search for “systematic review” on 2025-07-03, then filtered (peer-reviewed, OA, DOI, plausible page counts). PDFs parsed to structured text; SR metadata (databases, queries, criteria, RQs, etc.) extracted and manually checked. References extracted with a GROBID + AnyStyle pipeline; metadata resolved via Crossref, OpenAlex, Semantic Scholar, PubMed/Europe PMC; abstracts added where available. File Inventory sr4cs.json — SR entries with reported queries and linkage to reference IDs sr4cs_with_sql.json — SR entries plus sqlite_refined_queries (rewritten Boolean queries for SQLite FTS5) refs.db — SQLite database with reference metadata (FTS5 indexed) refs.ndjson — Elasticsearch bulk dump of references (ready for import) refs.parquet — Tabular reference metadata (not indexed) Known Limitations Reference coverage is large but not exhaustive; some entries are title-only. Translated SQL queries approximate original database behavior; engine differences (tokenization, indexing, operators) can affect results. FAIR & Documentation Findable: DOI, searchable metadata, stable identifiers (ref_id). Accessible: Public files (JSON/SQLite/NDJSON). Interoperable: Common schemas; JSON/SQL; ES bulk format. Reusable: Development code and usage examples. Contact Anonym