openai/webgpt_comparisons|自然语言处理数据集|问答系统数据集

---
pretty_name: WebGPT Comparisons
---
# Dataset Card for WebGPT Comparisons
## Dataset Description

In the [WebGPT paper](https://arxiv.org/abs/2112.09332), the authors trained a reward model from human feedback.
They used the reward model to train a long form question answering model to align with human preferences.
This is the dataset of all comparisons that were marked as suitable for reward modeling by the end of the WebGPT project.
There are 19,578 comparisons in total.

Each example in the dataset contains a pair of model answers for a question, and the associated metadata.
Each answer has a preference score from humans that can be used to determine which of the two answers are better.
Overall, an example has the following fields:

* `question`: The text of the question, together with the name of the dataset from which it was taken and a unique ID.
* `quotes_0`: The extracts that the model found while browsing for `answer_0`, together with the title of the page on which the extract was found, constructed from the HTML title and domain name of the page.
* `answer_0`: The final answer that the model composed using `quotes_0`.
* `tokens_0`: The prefix that would have been given to the model in the final step of the episode to create `answer_0`, and the completion given by the model or human. The prefix is made up of the question and the quotes, with some truncation, and the completion is simply the answer. Both are tokenized using the GPT-2 tokenizer. The concatenation of the prefix and completion is the input used for reward modeling.
* `score_0`: The strength of the preference for `answer_0` over `answer_1` as a number from −1 to 1. It sums to 0 with `score_1`, and an answer is preferred if and only if its score is positive. For reward modeling, we treat scores of 0 as soft 50% labels, and all other scores as hard labels (using only their sign).
* `quotes_1`: The counterpart to `quotes_0`.
* `answer_1`: The counterpart to `answer_0`.
* `tokens_1`: The counterpart to `tokens_0`.
* `score_1`: The counterpart to `score_0`.

This information was found in Appendix K of the WebGPT paper.

## Citation Information

[https://arxiv.org/abs/2112.09332](https://arxiv.org/abs/2112.09332)

```
@inproceedings{nakano2021webgpt,
author = {Reiichiro Nakano and Jacob Hilton and Suchir Balaji and Jeff Wu and Long Ouyang and Christina Kim and Christopher Hesse and Shantanu Jain and Vineet Kosaraju and William Saunders and Xu Jiang and Karl Cobbe and Tyna Eloundou and Gretchen Krueger and Kevin Button and Matthew Knight and Benjamin Chess and John Schulman},
title = {WebGPT: Browser-assisted question-answering with human feedback},
booktitle = {arXiv},
year = 2021,
}
```

Dataset added to the Hugging Face Hub by [@Tristan](https://huggingface.co/Tristan) and [@natolambert](https://huggingface.co/natolambert)

原始信息汇总

数据集卡片 for WebGPT Comparisons

数据集描述

该数据集包含WebGPT项目中所有被标记为适合奖励建模的比较。总共有19,578个比较。每个示例包含一对针对某个问题的模型答案及其相关元数据。每个答案都有一个来自人类的偏好分数，用于确定两个答案中哪个更好。

每个示例包含以下字段：

• question：问题的文本，以及问题来源的数据集名称和唯一ID。
• quotes_0：模型在浏览以生成answer_0时找到的摘录，以及摘录所在页面的标题（由页面的HTML标题和域名构成）。
• answer_0：模型使用quotes_0编写的最终答案。
• tokens_0：在生成answer_0的最后一步中会提供给模型的前缀，以及模型或人类给出的完成。前缀由问题和引用组成，并进行了一些截断，完成部分就是答案。两者都使用GPT-2分词器进行分词。前缀和完成的连接是用于奖励建模的输入。
• score_0：answer_0相对于answer_1的偏好强度，取值范围为-1到1。它与score_1之和为0，当且仅当其分数为正时，答案被偏好。对于奖励建模，我们将分数为0视为软50%标签，所有其他分数视为硬标签（仅使用其符号）。
• quotes_1：与quotes_0对应。
• answer_1：与answer_0对应。
• tokens_1：与tokens_0对应。
• score_1：与score_0对应。

这些信息来自WebGPT论文的附录K。