Data and code for: Schema-guided zero-shot decision models for chemical synthesis argument roles: comparison with supervised ChemBERT

This dataset provides research data and code for comparing schema-guided zero-shot chemical argument-role decisions with a released OSPAR-supervised ChemBERT classifier. The evaluated decision models are Jev 1.13.0, Clef and Clef-flash. The primary task assigns action-specific roles to supplied operation-candidate mention pairs, including a NONE label for unrelated candidates; mention detection and joint relation extraction are evaluated separately for ChemBERT. The Creative Commons (CC) reference contains 23 synthesis excerpts: 17 main evaluation procedures and six separate pilot procedures. The main evaluation comprises 226 scorable mentions, 288 resolved candidate pairs and 167 positive relations. The pilots add 72 candidate pairs and 52 positive relations. The archive contains 1440 original four-model prediction records. It preserves the frozen primary gold reference (v1) and a separately reported author-corrected reference (v1.1), which adds one temperature relation in CC004. Candidate pairs and model predictions are identical between these gold versions. A separate retrospective extension contains 1638 additional model-by-pair records for Jev, Clef and the same released ChemBERT: 258 pairs from 11 OSPAR test procedures and one fresh repeat of the 288 main CC pairs. The OSPAR procedures had been viewed during earlier prompt development and therefore do not constitute an untouched validation set. The fresh CC repeat uses the same 17 documents and is not an additional independent sample. The archive includes CC source excerpts and attribution; gold annotations, synthesis scopes, sentence boundaries and uncertainty masks; saved labels and probabilities; rolesets, fixed prompt definitions and request-building code; model manifests and artifact hashes; scoring events; and offline reproduction scripts. Inference audit logs contain 2304 CC native input/output records and 774 OSPAR output/provenance records, with model identifiers, timestamps and input hashes. Original OSPAR procedure text is not redistributed. The code reconstructs candidate sets and reproduces classification and extraction scores, document-bootstrap intervals, sensitivity analyses, descriptive annotation overlap and numerical plots from saved records. Offline score reproduction makes no model or network calls. Full inference replication additionally requires the relevant model weights or hosted-service access and the corresponding runtime/client implementations. Documentation is in English. Data, annotations and documentation are licensed under Creative Commons Attribution 4.0 International (CC BY 4.0). Author-created Python code is licensed under the MIT License. Original source attribution and third-party conditions are retained in the archive. A part of this work was further supported by the New Energy and Industrial Technology Development Organization (NEDO) Project to Develop Common Platforms Supporting the Widespread Use of Hydrogen. Preeti Yadav (NIMS) assisted with annotation work. Contact: Yuuya Nagata, [e-mail].