DOME Copilot - Data Files (V1)

To ensure your Zenodo upload is professional and easy to navigate, I have converted your text into a clean Markdown format. Zenodo’s description field supports Markdown, which will render the headings, lists, and bold text for better readability. DOME Copilot: Datasets Dataset Overview This dataset supports the manuscript on DOME Copilot, an AI for curating machine learning experiment metadata from scientific publications into sturtcured and transparent outputs. Archive Content: DOME_Copilot_Data_Package.zip The main data package is organized into the following directories and files: 1. Metadata Index DOME_Copilot_Data_Package_Metadata.csv: A comprehensive index mapping every PMCID (PubMed Central ID) in the dataset to the directories/datasets where its data can be found. Use this to quickly locate resources for a specific paper. 2. DOME Registry Database - Human Annotations DOME_Registry_Human_Reviews_258_20260205.json: The core dataset used to create the benchmark dataset, these are publications containing expert human-curated metadata annotations for the DOME Registry available papers - 258 entries at time of experimenation. 3. Copilot Output (AI Generated) Copilot_Processed_Datasets_JSON/: Contains the structured DOME JSON AI methods metadata generated by the DOME Copilot system. 222 Benchmark Cohort: Copilot_222_v0_Processed_2025-12-04 and its updated metadata. Copilot_222_v2_Processed_2026-03-02 and its updated metadata. 1012 Experimental Cohorts: Positive (ML papers): Copilot_1012_v2_Pos_Processed_2026-03-02, v0 outputs, and their updated metadata. Negative (Non-ML papers): Copilot_1012_v2_Neg_Processed_2026-03-02 and updated metadata. These directories represent different model iterations (v0 & v2) and contain the structured output data extracted directly from the publications. Note: we did not have a negative dataset for v0 experiments, so no v0 negative dataset available for these. 4. Main Text & Supplementary PDF Materials This archive contains the raw PDF full text and supplementary material downloaded from PMC for the study datasets, which serves as input data for the DOME Copilot analyses (Section 3). DOME_Registry_PMC_Supplementary/: Full text and Supplementary files for the 222 benchmark dataset articles of the DOME Registry (Benchmark Positive Set - Has Available Human Annotations to compare against). Positive_PMC_Supplementary/: Full text and Supplementary files for the 1012 "Positive" articles (Machine Learning relevant - Positive Set). Negative_PMC_Supplementary/: Full text and Supplementary files for the 1012 "Negative" articles (Non-ML relevant - Negative Set). 5. Evaluation Data 30_Evaluation_Source_JSONs_Human_and_Copilot_Including_PDFs/: The specific source files used for the deep-dive comparative evaluation (Human vs. AI) on a random subset but diversely weighted (journal/annotator) of 30 papers. Contains the Copilot JSONs, Human JSONs, and original PDFs used in the evaluation interface. Notes Matching: The DOME_Copilot_Data_Package_Metadata.csv file is the key to linking entries across these folders. Not all papers have supplementary data. Privacy: User-specific data (profiles, emails) has been excluded. PDFs: Full-text and supplementary PDFs are included in this bulk archive sourced from PMC through APIs in compliance with licensing for research. Identifiers - PMCID: PMCIDs are the common identifiers in use across folders.