Natural language processing systems for pathology parsing in limited data environments with uncertainty estimation

Objective: Cancer is a leading cause of death, but much of the diagnostic information is stored as unstructured data in pathology reports. We aim to improve uncertainty estimates of machine-learning based pathology parsers and evaluate performance in low data settings. Materials and Methods: Our data comes from the Urologic Outcomes Database at UCSF which includes 3,232 annotated prostate cancer pathology reports from 2001-2018. We approach 17 separate information extraction tasks, involving a wide range of pathologic features. To handle the diverse range of fields we required two statistical models, a document classification method for pathologic features with a small set of possible values and a token extraction method for pathologic features with a large set of values. For each model, we used isotonic calibration to improve the model’s estimates of its likelihood of being correct. Results: Our best document classifier method, a convolutional neural network, achieves a weighted ...Objective: 癌症是主要致死病因之一，但大量诊断信息以非结构化数据（unstructured data）的形式存储于病理报告中。本研究旨在提升基于机器学习的病理解析器的不确定性估计能力，并评估其在低数据量场景下的性能表现。 Materials and Methods: 本研究的数据来源于加州大学旧金山分校（UCSF）的泌尿外科结局数据库（Urologic Outcomes Database），该库涵盖2001年至2018年间的3232份经标注的前列腺癌病理报告。我们共实施17项独立的信息提取任务，涉及多种病理特征。为处理多样化的字段类型，我们采用了两种统计模型：针对可选值集合较小的病理特征，使用文档分类方法；针对可选值集合较大的病理特征，使用Token（词元）提取方法。针对每一种模型，我们均采用保序校准（isotonic calibration）以优化模型对自身预测正确性概率的估计。 Results: 本研究最优的文档分类方法——卷积神经网络（convolutional neural network），取得了加权……