DisKnE (Disease Knowledge Evaluation)

DisKnE 是基于 MedNLI 和 MEDIQA-NLI 构建的疾病知识评估基准。该基准旨在专门测试 ML 模型的医学推理能力，例如将症状映射到疾病。 该数据集是通过使用所需的医学推理类型注释每个正面 MedNLI 示例来构建的。负面例子是通过以对抗的方式破坏这些正面例子而产生的。此外，训练-测试拆分是根据疾病定义的，确保无法从训练数据中学习到有关测试疾病的知识。DisKnE is a disease knowledge evaluation benchmark constructed based on MedNLI and MEDIQA-NLI. This benchmark is specifically designed to test the medical reasoning capabilities of machine learning (ML) models, such as mapping symptoms to diseases. The dataset is built by annotating each positive MedNLI example with the required type of medical reasoning. Negative examples are generated by adversarially corrupting these positive examples. In addition, the train-test split is defined based on diseases, ensuring that no knowledge about the test diseases can be learned from the training data.

搜集汇总

背景与挑战

背景概述

DisKnE是一个基于MedNLI和MEDIQA-NLI构建的疾病知识评估基准，专门用于测试机器学习模型的医学推理能力，如症状到疾病的映射。该数据集通过注释正面示例并对抗性生成负面示例，训练和测试拆分基于疾病定义，以防止从训练数据中学习测试疾病的知识。