MultiClinAI Shared Task Training Set + MultiClinNER Test (Only Texts) and Background Sets

v1.2.1 2026/03/20: The texts for the MultiClinNER subtask's test and background set are now included in the files; they have been divided into batches to facilitate their processing. Batch 1 contains the test set + some background files, while batch 2 is made up entirely of the remaining background set files. The MultiClinAI (Multilingual Clinical Entity Annotation Projection and Extraction) shared task challenges participants to create systems that can automatically create multilingual versions of Gold Standard corpora from a seed language (in our case, Spanish) to six different target languages (Czech, Dutch, English, Italian, Romanian and Swedish). This is a process known as annotation projection. In parallel, participants are also challenged to create systems for clinical concept extraction in the seven languages of the task. This needs to be done for three different clinical information types: DISEASE, SYMPTOM and PROCEDURE. This repository currently includes the Training Data of the task, which includes part of the DisTEMIST, SympTEMIST and MedProcNER corpora, as well as the CardioCCC corpus (expanded from its original use in the MultiCardioNER shared task), in the seven languages mentioned above. The Test Set (only texts) and Background Set for the MultiClinNER subtask are also included. For more information about the creation process and context of this dataset, please visit the Data section of the task's website (linked below). As a note, each dataset in each language contains a slightly different version of the text due to translation revisions made during the annotation projection process, meaning they cannot be used for multilabel approaches. In short, the task is divided into two subtracks: - Sub-task MultiClinNER (Multilingual Comparable Clinical Entity Recognition). This is a common Named Entity Recognition task; using the texts in each language, try to extract the entities contained in each text (Spanish, Czech, Dutch, English, Italian, Romanian and Swedish). - Sub-task MultiClinCorpus (Multilingual Comparable Clinical Corpus Generation). This is an annotation projection task; starting from the original Spanish, try to obtain the annotations in the other languages (Czech, Dutch, English, Italian, Romanian and Swedish). Both tasks will be evaluated using common classification metrics: precision, recall and F-1. An official evaluation library will be released soon. Also in both sub-tasks, teams can submit results for any target language. Submitting for all languages is not mandatory. Participants are free to create their systems in any way they want (i.e. monolingual or multilingual models, word alignment or generative models, unilabel or multilabel, …), and the use of creative solutions is encouraged. File structure: MultiClinNER/├── MultiClinNER-es/│ ├── MultiClinNER-es-train/│ │ ├── MultiClinNER-es-train-disease/│ │ │ ├── MultiClinNER-es-train-disease-00001.txt│ │ │ ├── MultiClinNER-es-train-disease-00002.txt│ │ │ ├── MultiClinNER-es-train-disease-00003.txt│ │ │ ├── ...│ │ ├── MultiClinNER-es-train-symptom/│ │ │ ├── ...│ │ ├── MultiClinNER-es-train-procedure/│ │ │ ├── ...├── MultiClinNER-cz/│ ├── MultiClinNER-cz-train/│ │ ├── MultiClinNER-cz-train-disease/│ │ │ ├── MultiClinNER-cz-train-disease-00001.txt│ │ │ ├── ...│ │ ├── MultiClinNER-cz-train-symptom/│ │ │ ├── ...│ │ ├── MultiClinNER-cz-train-procedure/│ │ │ ├── ...├── MultiClinNER-{nl,en,it,ro,sv}/ (same structure as es and cz) ... (same for MultiClinCorpus folder) Resources Task Web Annotation guidelines Task Registration If you this dataset, please cite: @article{distemist2022overview,title={Overview of DisTEMIST at BioASQ: Automatic detection and normalization of diseases from clinical texts: results, methods, evaluation and multilingual resources},author={Miranda-Escalada, Antonio and Gascó, Luis and Lima-López, Salvador and Farré-Maduell, Eulàlia and Estrada, Darryl and Nentidis, Anastasios and Krithara, Anastasia and Katsimpras, Georgios and Paliouras, Georgios and Krallinger, Martin},booktitle={Working Notes of Conference and Labs of the Evaluation (CLEF) Forum. CEUR Workshop Proceedings},year={2022}} @inproceedings{symptemist2023overview, title={Overview of SympTEMIST at BioCreative VIII: corpus, guidelines and evaluation of systems for the detection and normalization of symptoms, signs and findings from text}, author={Lima-L{\'o}pez, Salvador and Farr{\'e}-Maduell, Eul{\`a}lia and Gasco-S{\'a}nchez, Luis and Rodr{\'\i}guez-Miret, Jan and Krallinger, Martin}} @inproceedings{medprocner2023overview,title={{Overview of MedProcNER task on medical procedure detection and entity linking at BioASQ 2023}},author={Lima-L{\’o}pez, Salvador and Farr{\’e}-Maduell, Eul{\`a}lia and Gasc{\’o}, Luis and Nentidis, Anastasios and Krithara, Anastasia and Katsimpras, Georgios and Paliouras, Georgios and Krallinger, Martin},booktitle={{Working Notes of CLEF 2023 – Conference and Labs of the Evaluation Forum}},year={2023}} @inproceedings{multicardioner2024overview,title={{Overview of MultiCardioNER task at BioASQ 2024 on Medical Speciality and Language Adaptation of Clinical NER Systems for Spanish, English and Italian}},author={Salvador Lima-López and Eulàlia Farré-Maduell and Jan Rodríguez-Miret and Miguel Rodríguez-Ortega and Livia Lilli and Jacopo Lenkowicz and Giovanna Ceroni and Jonathan Kossoff and Anoop Shah and Anastasios Nentidis and Anastasia Krithara and Georgios Katsimpras and Georgios Paliouras and Martin Krallinger},booktitle={CLEF Working Notes },year={2024},editor = {Faggioli, Guglielmo and Ferro, Nicola and Galuščáková, Petra and García Seco de Herrera, Alba}} Additional resources and corpora At the NLP for Biomedical Information Analysis group (formerly Text Mining Unit), one of our missions is the open publication of datasets to train and benchmark biomedical information extraction, normalization and indexing systems. For that reason, we have released multiple datasets as part of shared tasks over the years. If you are interested in MultiClinAI, you might want to take a look at some of our resources and competitions about: Clinical content extraction: DisTEMIST (diseases), MedProcNER/ProcTEMIST (clinical procedures), SympTEMIST (signs and findings), CANTEMIST (tumour morphology), CodiEsp (coding to ICD), PharmaCoNER (chemicals and proteins), LivingNER (species and humans), MultiCardioNER (diseases and medications, includes the DrugTEMIST corpus as well as cardiology-specific data) Socio-demographic / Social Determinants of Health content extraction: MEDDOPLACE (locations and more) MEDDOCAN (sensitive data), MEDDOPROF (occupations), ToxHabits (extraction of substance use-related content) Information extraction in social media: SocialDisNER (diseases), ProfNER (occupations) Linguistic aspects: BARR1 and BARR2 (abbreviation resolution) Machine Translation: ClinSpEn (EN<->ES clinical content translation) Summarization: MultiClinSUM (multilingual summarization of clinical content) Contact: - Salvador Lima-López (<salvador [dot] limalopez [at] gmail [dot] com>)- Martin Krallinger (<krallinger [dot] martin [at] gmail [dot] com>) License: This work is licensed under a Creative Commons Attribution 4.0 International License.版本v1.2.1，发布于2026年3月20日：本次更新已将MultiClinNER子任务的测试集与背景集文本纳入文件中，并按批次拆分以方便处理。批次1包含测试集与部分背景文件，批次2则仅包含剩余的背景集文件。 MultiClinAI（多语言临床实体标注投影与提取，Multilingual Clinical Entity Annotation Projection and Extraction）共享任务要求参与者构建可自动将金标准语料库从源语言（本任务中为西班牙语）投影至6种目标语言（捷克语、荷兰语、英语、意大利语、罗马尼亚语及瑞典语）的多语言版本的系统，该过程被称为标注投影。与此同时，参与者还需针对任务涵盖的7种语言构建临床概念抽取系统，需覆盖3类临床信息类型：疾病（DISEASE）、症状（SYMPTOM）与操作（PROCEDURE）。 本仓库目前包含本次任务的训练数据，涵盖上述7种语言下的DisTEMIST、SympTEMIST、MedProcNER语料库的部分数据，以及从MultiCardioNER共享任务原始用途扩展而来的CardioCCC语料库。此外还包含MultiClinNER子任务的测试集（仅文本）与背景集。如需了解本数据集的构建流程与背景信息，请访问任务官网的数据板块（链接见下方）。需注意的是，由于标注投影过程中进行了翻译修订，每种语言的数据集文本版本存在细微差异，因此不可用于多标签学习方法。 简言之，本次任务分为两个子赛道： - 子任务MultiClinNER（多语言可比临床实体识别，Multilingual Comparable Clinical Entity Recognition）：这是经典的命名实体识别（Named Entity Recognition, NER）任务，需利用各语言的文本，抽取文本中包含的实体（覆盖西班牙语、捷克语、荷兰语、英语、意大利语、罗马尼亚语及瑞典语）。 - 子任务MultiClinCorpus（多语言可比临床语料库生成，Multilingual Comparable Clinical Corpus Generation）：这是标注投影任务，需以原始西班牙语为起点，获取其余6种目标语言（捷克语、荷兰语、英语、意大利语、罗马尼亚语及瑞典语）的标注。 两项任务均将采用通用分类指标进行评估：精确率（precision）、召回率（recall）与F1值（F-1）。官方评估工具包即将发布。在两个子任务中，团队可选择任意目标语言提交结果，无需提交所有语言的结果。参与者可自由选择任意方式构建系统（例如单语言模型、多语言模型、词对齐或生成式模型、单标签或多标签方法等），鼓励采用创新性解决方案。 ### 文件结构 MultiClinNER/├── MultiClinNER-es/│ ├── MultiClinNER-es-train/│ │ ├── MultiClinNER-es-train-disease/│ │ │ ├── MultiClinNER-es-train-disease-00001.txt│ │ │ ├── MultiClinNER-es-train-disease-00002.txt│ │ │ ├── MultiClinNER-es-train-disease-00003.txt│ │ │ ├── ...│ │ ├── MultiClinNER-es-train-symptom/│ │ │ ├── ...│ │ ├── MultiClinNER-es-train-procedure/│ │ │ ├── ...├── MultiClinNER-cz/│ ├── MultiClinNER-cz-train/│ │ ├── MultiClinNER-cz-train-disease/│ │ │ ├── MultiClinNER-cz-train-disease-00001.txt│ │ │ ├── ...│ │ ├── MultiClinNER-cz-train-symptom/│ │ │ ├── ...│ │ ├── MultiClinNER-cz-train-procedure/│ │ │ ├── ...├── MultiClinNER-{nl,en,it,ro,sv}/（结构与es、cz一致） ...（MultiClinCorpus文件夹结构同理） ### 资源 - 任务官网 - 标注指南 - 任务报名 若使用本数据集，请引用以下文献： bibtex @article{distemist2022overview,title={Overview of DisTEMIST at BioASQ: Automatic detection and normalization of diseases from clinical texts: results, methods, evaluation and multilingual resources},author={Miranda-Escalada, Antonio and Gascó, Luis and Lima-López, Salvador and Farré-Maduell, Eulàlia and Estrada, Darryl and Nentidis, Anastasios and Krithara, Anastasia and Katsimpras, Georgios and Paliouras, Georgios and Krallinger, Martin},booktitle={Working Notes of Conference and Labs of the Evaluation (CLEF) Forum. CEUR Workshop Proceedings},year={2022}} @inproceedings{symptemist2023overview, title={Overview of SympTEMIST at BioCreative VIII: corpus, guidelines and evaluation of systems for the detection and normalization of symptoms, signs and findings from text}, author={Lima-López, Salvador and Farré-Maduell, Eulàlia and Gasco-Sánchez, Luis and Rodríguez-Miret, Jan and Krallinger, Martin}} @inproceedings{medprocner2023overview,title={{Overview of MedProcNER task on medical procedure detection and entity linking at BioASQ 2023}},author={Lima-López, Salvador and Farré-Maduell, Eulàlia and Gascó, Luis and Nentidis, Anastasios and Krithara, Anastasia and Katsimpras, Georgios and Paliouras, Georgios and Krallinger, Martin},booktitle={{Working Notes of CLEF 2023 – Conference and Labs of the Evaluation Forum}},year={2023}} @inproceedings{multicardioner2024overview,title={{Overview of MultiCardioNER task at BioASQ 2024 on Medical Speciality and Language Adaptation of Clinical NER Systems for Spanish, English and Italian}},author={Salvador Lima-López and Eulàlia Farré-Maduell and Jan Rodríguez-Miret and Miguel Rodríguez-Ortega and Livia Lilli and Jacopo Lenkowicz and Giovanna Ceroni and Jonathan Kossoff and Anoop Shah and Anastasios Nentidis and Anastasia Krithara and Georgios Katsimpras and Georgios Paliouras and Martin Krallinger},booktitle={CLEF Working Notes },year={2024},editor = {Faggioli, Guglielmo and Ferro, Nicola and Galuščáková, Petra and García Seco de Herrera, Alba}} ### 额外资源与语料库 生物医学信息分析自然语言处理工作组（前身为文本挖掘单元）的使命之一是公开发布用于训练和基准测试生物医学信息抽取、归一化与索引系统的数据集。因此，我们多年来已在多个共享任务中发布了多款数据集。若您对MultiClinAI感兴趣，可参考以下相关资源与竞赛： 1. **临床内容抽取**：DisTEMIST（疾病）、MedProcNER/ProcTEMIST（临床操作）、SympTEMIST（体征与发现）、CANTEMIST（肿瘤形态学）、CodiEsp（ICD编码）、PharmaCoNER（化学物质与蛋白质）、LivingNER（物种与人类）、MultiCardioNER（疾病与药物，包含DrugTEMIST语料库及心脏病学专用数据） 2. **社会人口统计学/健康社会决定因素内容抽取**：MEDDOPLACE（地理位置及其他）、MEDDOCAN（敏感数据）、MEDDOPROF（职业信息）、ToxHabits（物质使用相关内容抽取） 3. **社交媒体信息抽取**：SocialDisNER（疾病）、ProfNER（职业信息） 4. **语言学相关**：BARR1与BARR2（缩写消歧） 5. **机器翻译**：ClinSpEn（英语-西班牙语临床内容互译） 6. **文本摘要**：MultiClinSUM（临床内容多语言摘要） ### 联系方式 - Salvador Lima-López（邮箱：salvador[dot]limalopez[at]gmail[dot]com） - Martin Krallinger（邮箱：krallinger[dot]martin[at]gmail[dot]com） ### 许可协议 本作品采用知识共享署名4.0国际许可协议（Creative Commons Attribution 4.0 International License）进行许可。