ReCO

ReCO，一个人工策划的关于 Opinion 的中文阅读理解数据集。 ReCO 中的问题是向商业搜索引擎发出的基于意见的查询。这些段落由众包提供，他们从检索到的文档中提取支持片段。最后，众包工作人员给出了一个抽象的是/否/不确定的答案。 ReCO 的发布包含 300k 个问题，据我们所知，这些问题是中文阅读理解中最大的。 ReCO的一个突出特点是，除了原始上下文段落之外，我们还提供了可以直接用于回答问题的支持证据。质量分析表明 ReCO 的挑战在于它需要各种类型的推理技能，例如因果推理、逻辑推理等。当前的 QA 模型在许多问题回答问题上表现非常出色，例如 BERT (Devlin et al. 2018），在该数据集上仅达到 77% 的准确率，大大落后于人类近 92% 的性能，表明 ReCO 对机器阅读理解提出了很好的挑战。

ReCO is a manually curated Chinese reading comprehension dataset focused on opinion-based content. The questions in ReCO are opinion-based queries submitted to commercial search engines. The context passages are crowdsourced, with annotators extracting supporting fragments from the retrieved documents. Finally, the crowdworkers provide abstract yes/no/uncertain answers. The released ReCO dataset contains 300k questions, which, to the best of our knowledge, is the largest existing Chinese reading comprehension dataset. A prominent feature of ReCO is that, in addition to the original context passages, we also provide supporting evidence that can be directly utilized to answer the questions. Quality analysis demonstrates that the core challenge of ReCO lies in its demand for diverse reasoning skills, including causal reasoning, logical reasoning, and more. Current QA models, which achieve outstanding performance across numerous question answering tasks (e.g., BERT (Devlin et al., 2018)), only attain an accuracy of 77% on this dataset, far lagging behind the human performance of nearly 92%. This indicates that ReCO poses a significant challenge for machine reading comprehension systems.

搜集汇总

背景与挑战

背景概述

ReCO是一个专注于意见型查询的中文阅读理解数据集，包含30万个问题，是目前最大的中文阅读理解数据集之一。其特点在于提供支持证据，要求模型具备高级推理能力，对现有模型如BERT提出了显著挑战。