Supplementary Material for: National Institute of Health Stroke Scale (NIHSS) Scoring Tool Utilizing Medical Records: A Single-Center Validation Study

Objective: The prospective National Institutes of Health Stroke Scale (NIHSS) assessment is the standard for evaluating stroke severity. This study aimed to develop and validate an enhanced retrospective NIHSS (r-NIHSS) scoring method that integrates structured keyword mapping and established neurological scales, the Glasgow Coma Scale (GCS) for consciousness and the Medical Research Council (MRC) scale for motor strength, to enable reliable estimation of stroke severity from medical records. Subjects and Methods: We developed a structured chart-based method to estimate NIHSS scores from electronic health records, incorporating the GCS, the MRC scale, and a predefined list of standardized clinical descriptors. Four NIHSS-certified, blinded raters retrospectively evaluated 50 acute ischemic stroke patients with prospectively recorded NIHSS scores. Inter-rater reliability for each domain was analyzed using weighted kappa K(w), and agreement between r-NIHSS and prospective NIHSS (p-NIHSS) scores was assessed with the intraclass correlation coefficient (ICC). Results: Inter-rater reliability was almost perfect (ICC = 0.99 [95% CI 0.99 – 0.99]). The highest agreement across raters was seen in the level of consciousness domain (K (w) = 0.99 [95% CI 0.95 – 1.00]). The total r-NIHSS scores demonstrated near-perfect agreement with p-NIHSS scores (ICC = 0.99 [95% CI 0.98 – 0.99]). Conclusion: The updated chart-based r-NIHSS method is feasible and highly reliable for assessing stroke severity using routine medical documentation. This tool offers a practical, standardized approach for retrospective and registry-based studies where prospective NIHSS assessments are unavailable.