ChronoClinician: A Temporal Drug-Event Reasoning Benchmark for Clinical NLP

Temporal reasoning in clinical notes remains a critical unsolved challenge for medical natural language processing. Failure to correctly identify the chronological order of drug administrations and clinical events directly contributes to medication reconciliation errors, affecting thousands of hospitalized patients annually. In this work, we introduce ChronoClinician-1007—a scaled, multi-specialty temporal drug-event reasoning benchmark comprising 1,007 programmatically generated question-answer pairs spanning 33 medical specialties, stratified across three core relations: BEFORE, AFTER, and SIMULTANEOUS. We conduct a systematic evaluation across four distinct paradigms: ClinicalBERT, zero-shot DistilBERT-MNLI, zero-shot BART-Large-MNLI, and task-specific fine-tuned BioBERT. Our baseline evaluations expose an acute architectural blindness in pre-trained and zero-shot models regarding multi-drug co-administrations, with zero-shot architectures failing to surpass a 5.0% accuracy floor on concurrent timelines. Conversely, fine-tuned BioBERT achieves an unpenalized 100.0% accuracy under standard conditions. To determine model robustness, we introduce an adversarial "Hard Evaluation" protocol via the comprehensive redaction of surface temporal lexicons, alongside a rigorous cross-specialty domain generalization study across completely unseen hospital wards. The results expose a stark class asymmetry: under adversarial stress, directional logic (BEFORE accuracy collapsing to 0.0% standard, 64.0% cross-domain) is highly brittle and trigger-word dependent, whereas co-administration reasoning (SIMULTANEOUS) demonstrates flawless, out-of-distribution robustness (100.0% accuracy). This demonstrates that while directional chronology mimics simple surface patterns, concurrent logic successfully captures invariant clinical structures, clearing out the risk of specialty data leakage and charting a path forward for clinical commonsense modeling.