michaelszx/StepGame

---
license: mit
---

## Dataset Description
We have three files in the dataset (`k` is the number of maximum hops required to answer the question in the dataset):
- `train.json`: The "TrainVersion" is utilised in the baseline models presented in our paper. We use k=1,2,3,4,5 for training without noise.
- `valid.json`: The "TrainVersion" is utilised in the baseline models presented in our paper. We use k=1,2,3,4,5 for validation without noise.
- `test.json`: The "TrainVersion" is utilised in the baseline models presented in our paper. We use k=1,2,3,4,5,6,7,8,9,10 for testing with noise.

## Dataset Feature
In StepGame dataset, we have 4 features:
- story: A list of strings that describe the spatial relations between the agents.
- question: A string that asks a question about the spatial relations between two agents.
- label: A string that describes the spatial relation between the agents.
- k_hop: A string that describes the number of hops required to answer the question.

## Dataset Example
Here is an example of a sample from the dataset:
```
{
"story": [
"S is above J and to the left of J.",
"J is diagonally above B to the right at a 45 degree.",
"V is there and A is at the 2 position of a clock face."
],
"question": "What is the relation of the agent B to the agent J?",
"label": "lower-left",
"k_hop": "1"
}
```

## Source:
This dataset is sourced from the paper "StepGame: A New Benchmark for Robust Multi-Hop Spatial Reasoning in Texts" by Zhengxiang Shi, Qiang Zhang, and Aldo Lipani. The dataset is available at https://github.com/ZhengxiangShi/StepGame.

## Reference:
```
@inproceedings{stepGame2022shi,
title={StepGame: A New Benchmark for Robust Multi-Hop Spatial Reasoning in Texts},
author={Shi, Zhengxiang and Zhang, Qiang and Lipani, Aldo},
volume={36},
url={https://ojs.aaai.org/index.php/AAAI/article/view/21383},
DOI={10.1609/aaai.v36i10.21383},
booktitle={Proceedings of the AAAI Conference on Artificial Intelligence},
year={2022},
month={Jun.},
pages={11321-11329}
}
```

原始信息汇总

数据集描述

数据集包含三个文件：

• train.json: 用于论文中的基线模型训练，使用 k=1,2,3,4,5 进行训练，无噪声。
• valid.json: 用于论文中的基线模型验证，使用 k=1,2,3,4,5 进行验证，无噪声。
• test.json: 用于论文中的基线模型测试，使用 k=1,2,3,4,5,6,7,8,9,10 进行测试，有噪声。

数据集特征

StepGame 数据集包含以下四个特征：

• story: 描述代理之间空间关系的字符串列表。
• question: 询问两个代理之间空间关系的字符串。
• label: 描述代理之间空间关系的字符串。
• k_hop: 描述回答问题所需跳数（hops）的字符串。

数据集示例

以下是数据集中的一个样本示例： json { "story": [ "S is above J and to the left of J.", "J is diagonally above B to the right at a 45 degree.", "V is there and A is at the 2 position of a clock face." ], "question": "What is the relation of the agent B to the agent J?", "label": "lower-left", "k_hop": "1" }