An Annotated Dataset and Benchmark for Stroke Named Entity Recognition in Indonesian Patient Health Queries

Stroke is the leading cause of death and disability in Indonesia, yet clinical NLP tools for stroke triage support remain limited by the lack of annotated resources for patient-generated health text. We introduce StrokeID-NER, a stroke-specific named entity recognition dataset of 1,742 informal Indonesian patient queries annotated with 6,765 spans across four clinically grounded entity types: Symptom, Diagnosis, Temporal and Risk Factor. Inter-annotator agreement on 103 doubly annotated queries reached an entity-level F1 of 0.78 for exact spans and 0.96 for overlapping spans. We benchmark ten systems over three random seeds with query-level bootstrap confidence intervals: monolingual and multilingual transformers with linear and CRF decoders, and GPT-5.4 with zero-shot and five-shot prompting. XLM-RoBERTa-large with additional Indonesian NER training and a CRF decoder achieves the best macro F1 (0.792 ± 0.003). CRF decoding improves every encoder, whereas multilingual pretraining alone does not outperform monolingual pretraining of the same size. Five-shot GPT-5.4 (0.724) matches a fine-tuned base-size model but remains 6.8 points below the best system, with the largest gap on informal symptom expressions. StrokeID-NER, the annotation guideline, code and predictions are released publicly at https://doi.org/10.5281/zenodo.23910283