Annotation dataset of problematic opioid use and related contexts from MIMIC-III Critical Care Database discharge summaries

Opioid use disorder (OUD) is underdiagnosed in health system settings,
limiting research on OUD using electronic health records (EHRs). Medical
encounter notes can enrich structured EHR data with documented signs and
symptoms of OUD and social risks and behaviors. To capture this information at
scale, natural language processing tools must be developed and evaluated. We
conducted a pilot study that aimed to 1) develop and apply an annotation
schema to deeply characterize OUD and related clinical, behavioral, and
environmental factors; and 2) automate the annotation schema using machine
learning and deep learning-based approaches. De-identified patient data for
this study included hospital discharge summaries of patients with
_International Classification of Diseases_ (ICD-9) OUD diagnostic codes,
obtained from the MIMIC-III Critical Care Database. We developed an annotation
schema to characterize problematic opioid use, identify individuals with
potential OUD, and provide psychosocial context. The final annotation schema
contained 33 classes. Two annotators reviewed discharge summaries from a
random sample of 100 of these patients. The first corpus of 40 patients was
reviewed by both annotators. We achieved moderate inter-annotator agreement,
with F1-scores across all classes increasing from 48% to 66%. The second
corpus of 60 patients was reviewed by a single annotator. The shared database
contains the resulting 3,270 annotations with the note identifier, span offset
with accompanying text snippet, and class assignments and may be useful to
future development of natural language processing systems related to OUD.