CommonsenseQA

CommonsenseQA 是常识问答任务的数据集。该数据集由 12,247 个问题组成，每个问题有 5 个选项。数据集是由 Amazon Mechanical Turk 工作人员在以下过程中生成的（括号中提供了一个示例）：人群工作人员从 ConceptNet（“河流”）和三个目标概念（“瀑布”、“桥梁”、“ Valley”）都通过相同的 ConceptNet 关系（“AtLocation”）相关联，工作人员提出三个问题，每个目标概念一个，这样只有特定的目标概念是答案，而其他两个干扰概念不是，（ “在河上哪里可以在阳光明媚的日子里端起杯子来接水？”，“我在哪里可以站在河上看水落而不湿？”，“我正在过河，我的脚是湿但我的身体是干的，我在哪里？”）对于每个问题，另一位工人从概念网（“卵石”、“溪流”、“银行”）中选择一个额外的干扰物（“卵石”、“溪流”、“银行”），作者选择另一个干扰物（“山”、 “底部”，“岛”）手动。

CommonsenseQA is a dataset for the commonsense question answering task. It consists of 12,247 questions, each with 5 answer options. The dataset was created by Amazon Mechanical Turk workers through the following process (with an example provided in parentheses): First, crowd workers selected a source concept from ConceptNet (e.g., "river") and three target concepts, all of which are associated with the source concept via the same ConceptNet relation, "AtLocation": "waterfall", "bridge", and "valley". Next, they formulated three questions, one for each target concept, such that only the specific target concept served as the correct answer, while the other two were initial distractor concepts. The example questions are: "Where on a river can you hold up a cup to catch water on a sunny day?", "Where can I stand on a river to watch falling water without getting wet?", and "I am crossing a river, my feet are wet but my body is dry—where am I?". For each question, a second worker selected an additional distractor from ConceptNet; example candidate distractors include "pebble", "stream", and "bank". Finally, the authors manually selected one more distractor for each question, with examples being "mountain", "bottom", and "island".

搜集汇总

背景与挑战

背景概述

CommonsenseQA是一个用于常识问答任务的数据集，包含12,247个问题，每个问题有5个选项。该数据集通过Amazon Mechanical Turk众包平台基于ConceptNet知识库生成，旨在评估模型对日常常识知识的理解能力，其问题设计确保答案唯一且干扰项合理。