HotpotQA

HotpotQA 是收集在英语维基百科上的问答数据集，包含大约 113K 众包问题，这些问题的构建需要两篇维基百科文章的介绍段落才能回答。数据集中的每个问题都带有两个黄金段落，以及这些段落中的句子列表，众包工作人员认为这些句子是回答问题所必需的支持事实。 HotpotQA 提供了多种推理策略，包括涉及问题中缺失实体的问题、交叉问题（什么满足属性 A 和属性 B？）和比较问题，其中两个实体通过一个共同属性进行比较等。在少文档干扰设置中，QA 模型有 10 个段落，保证能找到黄金段落；在开放域全维基设置中，模型只给出问题和整个维基百科。模型根据其答案准确性和可解释性进行评估，其中前者被测量为具有完全匹配 (EM) 和 unigram F1 的预测答案和黄金答案之间的重叠，后者关注预测的支持事实句子与人类注释的匹配程度（Supporting Fact EM/F1)。该数据集还报告了一个联合指标，它鼓励系统同时在两项任务上表现良好。

来源：通过迭代查询生成回答复杂的开放域问题

HotpotQA is a question answering (QA) dataset collected from English Wikipedia, containing approximately 113K crowdsourced questions. These questions can only be answered by leveraging the introductory paragraphs of two separate Wikipedia articles. Each question in the dataset is paired with two gold paragraphs, along with a list of sentences from these paragraphs that crowdworkers have identified as supporting facts necessary for answering the question. HotpotQA covers diverse reasoning strategies, including questions with missing entities in the query, cross-property questions (e.g., "what satisfies both property A and property B?"), and comparative questions where two entities are compared through a shared attribute, among others. In the few-document distraction setting, QA models are provided with 10 paragraphs, ensuring the gold paragraphs can be located; in the open-domain full Wikipedia setting, models are only given the question and the entire English Wikipedia corpus. Models are evaluated based on both answer accuracy and interpretability. The former is measured by the overlap between predicted answers and gold answers using Exact Match (EM) and unigram F1 scores, while the latter focuses on the degree of match between predicted supporting fact sentences and human-annotated ones (Supporting Fact EM/F1). The dataset also reports a joint metric that encourages systems to perform well on both tasks. Source: Answering Complex Open-Domain Questions via Iterative Query Generation