C3

C3 是一个自由形式的多选中文机器阅读理解数据集。我们展示了第一个自由形式的多选中文机器阅读理解数据集（C^3），包含 13,369 个文档（对话或更正式的混合体裁文本）及其相关的 19,577 个从中文收集的自由形式选择题-作为第二语言的考试。我们对这些现实世界问题所需的先验知识（即语言、特定领域和一般世界知识）进行了全面分析。我们实施了基于规则和流行的神经方法，发现性能最佳的模型 (68.5%) 和人类读者 (96.0%) 之间仍然存在显着的性能差距，尤其是在需要先验知识的问题上。我们进一步研究了基于英语翻译相关数据集的干扰物合理性和数据增强对模型性能的影响。我们预计 C^3 将对现有系统提出巨大挑战，因为回答 86.8% 的问题需要随附文档内外的知识，我们希望 C^3 可以作为研究如何利用各种先验知识的平台更好地理解给定的书面或口头文本。 C^3 可在 https://dataset.org/c3/ 获得。

C3 is a free-form multiple-choice Chinese machine reading comprehension dataset. We present the first free-form multiple-choice Chinese machine reading comprehension dataset (C^3), which includes 13,369 documents (dialogues or more formal mixed-genre texts) and their corresponding 19,577 free-form multiple-choice questions collected from Chinese as a second language examinations. We performed a comprehensive analysis of the prior knowledge required for these real-world questions, namely linguistic knowledge, domain-specific knowledge, and general world knowledge. We implemented rule-based and popular neural methods, and found that there remains a significant performance gap between the best-performing model (68.5%) and human readers (96.0%), especially for questions that require prior knowledge. We further studied the impact of distractor plausibility and data augmentation on model performance based on related English-translated datasets. We expect that C^3 will pose great challenges to existing systems, since answering 86.8% of the questions demands knowledge both within and outside the accompanying documents, and we hope that C^3 can act as a platform for researching how to utilize various prior knowledge to better understand the given written or oral texts. C^3 is accessible at https://dataset.org/c3/.

搜集汇总

背景与挑战

背景概述

C3是一个自由形式的多选中文机器阅读理解数据集，包含13,369个文档和19,577个选择题，源自中文作为第二语言的考试。其关键特点是86.8%的问题需要文档内外的先验知识（如语言、领域和世界知识），导致性能最佳的模型（68.5%）与人类读者（96.0%）之间存在显著差距，因此该数据集对现有系统构成巨大挑战，并可作为研究利用先验知识进行文本理解的平台。