ShAReCLEF eHealth Evaluation Lab 2014 (Task 2): Disorder Attributes in Clinical Reports

This project focuses on Task 2 of the 2014 ShARe/CLEF eHealth evaluation lab, which extended Task 1 of the 2013 ShARe/CLEF eHealth evaluation lab by focusing on identification and normalization of disorder attributes. The task comprises two subtasks: attribute normalization (task 2a) and cue identification (task 2b). We instructed participants to develop a system which either kept or updated a default attribute value for each task. Participant systems were evaluated against a blind reference standard of 133 discharge summaries using Accuracy (task 2a) and F-score (task 2b). This is the second year of the ShARe/CLEF eHealth Evaluation Lab, a shared task focused on natural language processing (NLP) and information retrieval (IR) for clinical care. The task is co-organized by the Shared Annotated Resources (ShARe) project and the CLEF Initiative (Conference and Labs of the Evaluation Forum, formerly known as Cross-Language Evaluation Forum). The vision of ShARe/CLEF is two-fold: (1) to develop tasks that potentially impact patient understanding of medical information and (2) to provide the community with an increasingly sophisticated dataset of clinical narrative to advance the state-of-the-art in Natural Language Processing, Information Extraction and Information Retrieval in healthcare.