StrategyQA

A Question Answering Benchmark with Implicit Reasoning Strategies

The StrategyQA dataset was created through a crowdsourcing pipeline for eliciting creative and diverse yes/no questions that require implicit reasoning steps. To solve questions in StrategyQA, the reasoning steps should be inferred using a strategy. To guide and evaluate the question answering process, each example in StrategyQA was annotated with a decomposition into reasoning steps for answering it, and Wikipedia paragraphs that provide evidence for the answer to each step.

Illustrated in the figure below: Questions in StrategyQA (Q1) require implicit reasoning, in contrast to multi-step questions that explicitly specify the reasoning process (Q2). Each training example contains a question (Q1), yes/no answer (A), decomposition (D), and evidence paragraphs (E).

[strategyqa_test](https://huggingface.co/datasets/voidful/StrategyQA/resolve/main/strategyqa_test.json)
[strategyqa_train](https://huggingface.co/datasets/voidful/StrategyQA/blob/main/strategyqa_train.json)
[strategyqa_train_filtered](https://huggingface.co/datasets/voidful/StrategyQA/blob/main/strategyqa_train_filtered.json)
[strategyqa_train_paragraphs](https://huggingface.co/datasets/voidful/StrategyQA/blob/main/strategyqa_train_paragraphs.json)

Paper

Title: Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategies

Authors: Mor Geva, Daniel Khashabi, Elad Segal, Tushar Khot, Dan Roth, Jonathan Berant

Transactions of the Association for Computational Linguistics (TACL), 2021

Citation:
```
@article{geva2021strategyqa,
title = {{Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategies}},
author = {Geva, Mor and Khashabi, Daniel and Segal, Elad and Khot, Tushar and Roth, Dan and Berant, Jonathan},
journal = {Transactions of the Association for Computational Linguistics (TACL)},
year = {2021},
}
```

# 带有隐式推理策略的问答基准数据集

策略问答（StrategyQA）数据集是通过众包流程构建的，旨在征集兼具创意性与多样性的是非类问题，此类问题需要依托隐式推理步骤才能解答。若要解决策略问答（StrategyQA）数据集中的问题，需通过特定推理策略推断出其隐含的推理步骤。为引导并评估问答过程，该数据集的每个样本均标注了用于解答对应问题的推理步骤分解，以及为每个推理步骤提供证据的维基百科段落。

如下图所示：策略问答（StrategyQA）中的问题（Q1）需要进行隐式推理，而与之相对的是明确指定推理过程的多步问题（Q2）。每个训练样本包含问题（Q1）、是非答案（A）、推理分解步骤（D）与证据段落（E）。

[strategyqa_test](https://huggingface.co/datasets/voidful/StrategyQA/resolve/main/strategyqa_test.json)
[strategyqa_train](https://huggingface.co/datasets/voidful/StrategyQA/blob/main/strategyqa_train.json)
[strategyqa_train_filtered](https://huggingface.co/datasets/voidful/StrategyQA/blob/main/strategyqa_train_filtered.json)
[strategyqa_train_paragraphs](https://huggingface.co/datasets/voidful/StrategyQA/blob/main/strategyqa_train_paragraphs.json)

## 论文
标题：《亚里士多德用过笔记本电脑吗？一个带有隐式推理策略的问答基准数据集》
作者：莫·格瓦（Mor Geva）、丹尼尔·哈沙比（Daniel Khashabi）、埃拉德·西格尔（Elad Segal）、图什·科特（Tushar Khot）、丹·罗斯（Dan Roth）、乔纳森·贝兰特（Jonathan Berant）
期刊：《计算语言学协会汇刊（Transactions of the Association for Computational Linguistics，TACL）》，2021年

引用：

@article{geva2021strategyqa,
title = {{Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategies}},
author = {Geva, Mor and Khashabi, Daniel and Segal, Elad and Khot, Tushar and Roth, Dan and Berant, Jonathan},
journal = {Transactions of the Association for Computational Linguistics (TACL)},
year = {2021},
}

搜集汇总

背景与挑战

背景概述

StrategyQA是一个需要隐含推理策略的问答基准数据集，包含通过众包创建的yes/no问题，每个问题都标注了推理步骤分解和证据段落，用于评估问答系统的推理能力。