Medical Expert Annotations of Unsupported Facts in Doctor-Written and LLM-Generated Patient Summaries

Large language models in healthcare can generate informative patient summaries
while reducing the documentation workload of healthcare professionals.
However, these models are prone to producing hallucinations, that is,
generating unsupported information, which is problematic in the sensitive
healthcare domain. To better characterize unsupported facts in medical texts,
we developed a rigorous labeling protocol. Following this protocol, two
medical experts annotated unsupported facts in 100 doctor-written summaries
from the MIMIC-IV-Note Discharge Instructions and hallucinations 100 LLM-
generated patient summaries. Here, we are releasing two datasets based on
these annotations: _Hallucinations-MIMIC-DI_ and _Hallucinations-Generated-
DI_. We find that using these datasets to train on hallucination-free examples
effectively reduces hallucinations for both Llama 2 (2.60 to 1.55
hallucinations per summary) and GPT-4 (0.70 to 0.40). Furthermore, we created
a preprocessed version of the MIMIC-IV-Notes Discharge Instructions, releasing
both a full-context version ( _MIMIC-IV-Note-Ext-DI_ ) and a version that only
uses the Brief Hospital Course for context ( _MIMIC-IV-Note-Ext-DI-BHC_ ).