KMIR

KMIR数据集由华为泊松实验室创建，旨在评估语言模型在知识记忆、识别和推理方面的能力。该数据集包含184,348个精心设计的问题，覆盖通用知识、特定领域知识和常识三大类。数据集通过从WikiData和ConceptNet等公开知识源收集的192,078个知识三元组生成。KMIR数据集的应用领域广泛，主要用于解决语言模型作为知识源的可靠性和有效性问题，推动语言模型在知识相关任务中的设计和优化。The KMIR Dataset was created by Huawei Poisson Lab, aiming to evaluate the capabilities of language models in knowledge memorization, recognition and reasoning. This dataset contains 184,348 meticulously designed questions covering three major categories: general knowledge, domain-specific knowledge and common sense. It is generated from 192,078 knowledge triples collected from public knowledge sources such as Wikidata and ConceptNet. The KMIR Dataset has broad application scenarios, and is mainly used to address the reliability and validity issues of language models acting as knowledge sources, so as to promote the design and optimization of language models in knowledge-related tasks.

原始信息汇总

KMIR: 语言模型知识记忆、识别和推理能力评估基准

数据集文件

• kmir_train_cloze.jsonl
• kmir_dev_cloze.jsonl
• kmir_test_cloze.jsonl

搜集汇总

背景与挑战

背景概述

KMIR是一个用于评估语言模型知识记忆、识别和推理能力的基准数据集。数据集以压缩文件（kmir.zip）形式提供，包含训练、开发和测试三个JSONL格式文件，专门设计用于语言模型性能的全面评估。