openbookqa

# Dataset Card for OpenBookQA

## Table of Contents
- [Dataset Description](#dataset-description)
- [Dataset Summary](#dataset-summary)
- [Supported Tasks and Leaderboards](#supported-tasks-and-leaderboards)
- [Languages](#languages)
- [Dataset Structure](#dataset-structure)
- [Data Instances](#data-instances)
- [Data Fields](#data-fields)
- [Data Splits](#data-splits)
- [Dataset Creation](#dataset-creation)
- [Curation Rationale](#curation-rationale)
- [Source Data](#source-data)
- [Annotations](#annotations)
- [Personal and Sensitive Information](#personal-and-sensitive-information)
- [Considerations for Using the Data](#considerations-for-using-the-data)
- [Social Impact of Dataset](#social-impact-of-dataset)
- [Discussion of Biases](#discussion-of-biases)
- [Other Known Limitations](#other-known-limitations)
- [Additional Information](#additional-information)
- [Dataset Curators](#dataset-curators)
- [Licensing Information](#licensing-information)
- [Citation Information](#citation-information)
- [Contributions](#contributions)

## Dataset Description

- **Homepage:** [https://allenai.org/data/open-book-qa](https://allenai.org/data/open-book-qa)
- **Repository:** [More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
- **Paper:** [More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
- **Point of Contact:** [More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
- **Size of downloaded dataset files:** 2.89 MB
- **Size of the generated dataset:** 2.88 MB
- **Total amount of disk used:** 5.78 MB

### Dataset Summary

OpenBookQA aims to promote research in advanced question-answering, probing a deeper understanding of both the topic
(with salient facts summarized as an open book, also provided with the dataset) and the language it is expressed in. In
particular, it contains questions that require multi-step reasoning, use of additional common and commonsense knowledge,
and rich text comprehension.
OpenBookQA is a new kind of question-answering dataset modeled after open book exams for assessing human understanding of
a subject.

### Supported Tasks and Leaderboards

[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)

### Languages

[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)

## Dataset Structure

### Data Instances

#### main

- **Size of downloaded dataset files:** 1.45 MB
- **Size of the generated dataset:** 1.45 MB
- **Total amount of disk used:** 2.88 MB

An example of 'train' looks as follows:
```
{'id': '7-980',
'question_stem': 'The sun is responsible for',
'choices': {'text': ['puppies learning new tricks',
'children growing up and getting old',
'flowers wilting in a vase',
'plants sprouting, blooming and wilting'],
'label': ['A', 'B', 'C', 'D']},
'answerKey': 'D'}
```

#### additional

- **Size of downloaded dataset files:** 1.45 MB
- **Size of the generated dataset:** 1.45 MB
- **Total amount of disk used:** 2.88 MB

An example of 'train' looks as follows:
```
{'id': '7-980',
'question_stem': 'The sun is responsible for',
'choices': {'text': ['puppies learning new tricks',
'children growing up and getting old',
'flowers wilting in a vase',
'plants sprouting, blooming and wilting'],
'label': ['A', 'B', 'C', 'D']},
'answerKey': 'D',
'fact1': 'the sun is the source of energy for physical cycles on Earth',
'humanScore': 1.0,
'clarity': 2.0,
'turkIdAnonymized': 'b356d338b7'}
```

### Data Fields

The data fields are the same among all splits.

#### main
- `id`: a `string` feature.
- `question_stem`: a `string` feature.
- `choices`: a dictionary feature containing:
- `text`: a `string` feature.
- `label`: a `string` feature.
- `answerKey`: a `string` feature.

#### additional
- `id`: a `string` feature.
- `question_stem`: a `string` feature.
- `choices`: a dictionary feature containing:
- `text`: a `string` feature.
- `label`: a `string` feature.
- `answerKey`: a `string` feature.
- `fact1` (`str`): oOriginating common knowledge core fact associated to the question.
- `humanScore` (`float`): Human accuracy score.
- `clarity` (`float`): Clarity score.
- `turkIdAnonymized` (`str`): Anonymized crowd-worker ID.

### Data Splits

| name | train | validation | test |
|------------|------:|-----------:|-----:|
| main | 4957 | 500 | 500 |
| additional | 4957 | 500 | 500 |

## Dataset Creation

### Curation Rationale

[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)

### Source Data

#### Initial Data Collection and Normalization

[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)

#### Who are the source language producers?

[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)

### Annotations

#### Annotation process

[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)

#### Who are the annotators?

[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)

### Personal and Sensitive Information

[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)

## Considerations for Using the Data

### Social Impact of Dataset

[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)

### Discussion of Biases

[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)

### Other Known Limitations

[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)

## Additional Information

### Dataset Curators

[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)

### Licensing Information

[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)

### Citation Information

```
@inproceedings{OpenBookQA2018,
title={Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering},
author={Todor Mihaylov and Peter Clark and Tushar Khot and Ashish Sabharwal},
booktitle={EMNLP},
year={2018}
}

```

### Contributions

Thanks to [@thomwolf](https://github.com/thomwolf), [@patrickvonplaten](https://github.com/patrickvonplaten), [@lewtun](https://github.com/lewtun) for adding this dataset.

# 开放书籍问答数据集（OpenBookQA）数据集卡片

## 目录
- [数据集描述](#dataset-description)
- [数据集概述](#dataset-summary)
- [支持任务与排行榜](#supported-tasks-and-leaderboards)
- [语言](#languages)
- [数据集结构](#dataset-structure)
- [数据实例](#data-instances)
- [数据字段](#data-fields)
- [数据划分](#data-splits)
- [数据集构建](#dataset-creation)
- [遴选依据](#curation-rationale)
- [源数据](#source-data)
- [标注](#annotations)
- [个人与敏感信息](#personal-and-sensitive-information)
- [数据集使用注意事项](#considerations-for-using-the-data)
- [数据集的社会影响](#social-impact-of-dataset)
- [偏差讨论](#discussion-of-biases)
- [其他已知局限性](#other-known-limitations)
- [附加信息](#additional-information)
- [数据集策展人](#dataset-curators)
- [许可信息](#licensing-information)
- [引用信息](#citation-information)
- [贡献](#contributions)

## 数据集描述

- **主页**：[https://allenai.org/data/open-book-qa](https://allenai.org/data/open-book-qa)
- **代码仓库**：[更多信息待补充](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
- **相关论文**：[更多信息待补充](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
- **联系人**：[更多信息待补充](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
- **下载数据集文件大小**：2.89 MB
- **生成数据集大小**：2.88 MB
- **磁盘总占用空间**：5.78 MB

### 数据集概述

OpenBookQA（开放书籍问答数据集）旨在推动高级问答研究，深入探究模型对目标主题的理解——核心事实以“开放书籍”形式汇总并随数据集一同提供，同时也考察对问题表述语言的掌握程度。具体而言，该数据集包含需要多步推理、运用额外通用常识知识以及具备丰富文本理解能力的问答问题。OpenBookQA是一类新型问答数据集，其设计参考了用于评估人类对学科掌握程度的开卷考试模式。

### 支持任务与排行榜

[更多信息待补充](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)

### 语言

[更多信息待补充](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)

## 数据集结构

### 数据实例

#### main
- **下载数据集文件大小**：1.45 MB
- **生成数据集大小**：1.45 MB
- **磁盘总占用空间**：2.88 MB

以下是`train`划分的一个示例：

{'id': '7-980',
'question_stem': 'The sun is responsible for',
'choices': {'text': ['puppies learning new tricks',
'children growing up and getting old',
'flowers wilting in a vase',
'plants sprouting, blooming and wilting'],
'label': ['A', 'B', 'C', 'D']},
'answerKey': 'D'}

#### additional
- **下载数据集文件大小**：1.45 MB
- **生成数据集大小**：1.45 MB
- **磁盘总占用空间**：2.88 MB

以下是`train`划分的一个示例：

{'id': '7-980',
'question_stem': 'The sun is responsible for',
'choices': {'text': ['puppies learning new tricks',
'children growing up and getting old',
'flowers wilting in a vase',
'plants sprouting, blooming and wilting'],
'label': ['A', 'B', 'C', 'D']},
'answerKey': 'D',
'fact1': 'the sun is the source of energy for physical cycles on Earth',
'humanScore': 1.0,
'clarity': 2.0,
'turkIdAnonymized': 'b356d338b7'}

### 数据字段

所有划分的数据字段均保持一致。

#### main
- `id`：字符串类型特征。
- `question_stem`：字符串类型特征。
- `choices`：字典类型特征，包含：
- `text`：字符串类型特征。
- `label`：字符串类型特征。
- `answerKey`：字符串类型特征。

#### additional
- `id`：字符串类型特征。
- `question_stem`：字符串类型特征。
- `choices`：字典类型特征，包含：
- `text`：字符串类型特征。
- `label`：字符串类型特征。
- `answerKey`：字符串类型特征。
- `fact1`（`str`）：关联至该问题的核心常识事实。
- `humanScore`（`float`）：人类标注准确率得分。
- `clarity`（`float`）：问题清晰度得分。
- `turkIdAnonymized`（`str`）：匿名化的众包工作者ID。

### 数据划分

| 划分名称 | 训练集 | 验证集 | 测试集 |
|----------------|-------:|-------:|-------:|
| main | 4957 | 500 | 500 |
| additional | 4957 | 500 | 500 |

## 数据集构建

### 遴选依据

[更多信息待补充](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)

### 源数据

#### 初始数据收集与标准化

[更多信息待补充](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)

#### 源语言生产者是谁？

[更多信息待补充](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)

### 标注

#### 标注流程

[更多信息待补充](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)

#### 标注人员是谁？

[更多信息待补充](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)

### 个人与敏感信息

[更多信息待补充](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)

## 数据集使用注意事项

### 数据集的社会影响

[更多信息待补充](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)

### 偏差讨论

[更多信息待补充](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)

### 其他已知局限性

[更多信息待补充](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)

## 附加信息

### 数据集策展人

[更多信息待补充](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)

### 许可信息

[更多信息待补充](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)

### 引用信息

@inproceedings{OpenBookQA2018,
title={Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering},
author={Todor Mihaylov and Peter Clark and Tushar Khot and Ashish Sabharwal},
booktitle={EMNLP},
year={2018}
}

### 贡献
感谢 [@thomwolf](https://github.com/thomwolf)、[@patrickvonplaten](https://github.com/patrickvonplaten)、[@lewtun](https://github.com/lewtun) 为本数据集的添加提供支持。