Weekly supervised Multilingual Data Set to train Named Entity Recognition for Symptom Extraction

Data Sets were generated using the Weakly Supervised NER pipeline (https://github.com/HUMADEX/Weekly-Supervised-NER-pipline) to train the symptom extraction NER models. 

Supported Languages and dataset locations for the specific language:

    English (base language): https://huggingface.co/HUMADEX/english_medical_ner    German: https://huggingface.co/HUMADEX/german_medical_ner    Italian: https://huggingface.co/HUMADEX/italian_medical_ner    Spanish: https://huggingface.co/HUMADEX/spanish_medical_ner    Greek: https://huggingface.co/HUMADEX/german_medical_ner    Slovenian: https://huggingface.co/HUMADEX/slovenian_medical_ner    Polish: https://huggingface.co/HUMADEX/polish_medical_ner    Portuguese: https://huggingface.co/HUMADEX/portugese_medical_ner

Dataset Building 

Data Integration and Preprocessing

Data Cleaning

Annotation with Stanza's i2b2 Clinical Model 

Translation into the targeted language

Word Alignment 

Data Augmentation 

AcknowledgementThis dataset had been created as part of joint research of HUMADEX research group (https://www.linkedin.com/company/101563689/) and has received funding by the European Union Horizon Europe Research and Innovation Program project SMILE (grant number 101080923) and Marie Skłodowska-Curie Actions (MSCA) Doctoral Networks, project BosomShield ((rant number 101073222). Responsibility for the information and views expressed herein lies entirely with the authors.

Authors:dr. Izidor Mlakar, Rigona Sallauka, dr. Umut Arioz, dr. Matej Rojc