ShAReCLEF eHealth Evaluation Lab 2014 (Task 2): Disorder Attributes in Clinical Reports

This project focuses on Task 2 of the 2014 ShARe/CLEF eHealth evaluation lab, which extended Task 1 of the 2013 ShARe/CLEF eHealth evaluation lab by focusing on identification and normalization of disorder attributes. The task comprises two subtasks: attribute normalization (task 2a) and cue identification (task 2b). We instructed participants to develop a system which either kept or updated a default attribute value for each task. Participant systems were evaluated against a blind reference standard of 133 discharge summaries using Accuracy (task 2a) and F-score (task 2b). 

This is the second year of the ShARe/CLEF eHealth Evaluation Lab, a shared task focused on natural language processing (NLP) and information retrieval (IR) for clinical care. The task is co-organized by the Shared Annotated Resources (ShARe) project and the CLEF Initiative (Conference and Labs of the Evaluation Forum, formerly known as Cross-Language Evaluation Forum). The vision of ShARe/CLEF is two-fold: (1) to develop tasks that potentially impact patient understanding of medical information and (2) to provide the community with an increasingly sophisticated dataset of clinical narrative to advance the state-of-the-art in Natural Language Processing, Information Extraction and Information Retrieval in healthcare.

本项研究聚焦于2014年ShARe/CLEF eHealth评估实验室的任务2，该任务在2013年ShARe/CLEF eHealth评估实验室的任务1基础上进行扩展，主要关注疾病属性的识别与规范化。该任务包括两个子任务：属性规范化（任务2a）和线索识别（任务2b）。我们指导参与者开发一个系统，该系统能够在每个任务中保持或更新默认属性值。参与者的系统通过与133份出院总结的无视标准进行评估，评估指标包括准确率（任务2a）和F分数（任务2b）。这是ShARe/CLEF eHealth评估实验室的第二年，该实验室是一个以自然语言处理（NLP）和信息检索（IR）在临床护理中的应用为共同任务的研究项目。该任务由共享标注资源（ShARe）项目和CLEF倡议（Conference and Labs of the Evaluation Forum，原名Cross-Language Evaluation Forum）共同组织。ShARe/CLEF的愿景是双重的：（1）开发可能影响患者对医疗信息理解的任务；（2）为社区提供日益复杂的临床叙事数据集，以推动医疗保健领域自然语言处理、信息提取和信息检索的最新技术水平。