lovethayo/HelpSteer

---
license: cc-by-4.0
language:
- en
pretty_name: Helpfulness SteerLM Dataset
size_categories:
- 10K<n<100K
tags:
- human-feedback
dataset_info:
features:
- name: prompt
dtype: string
- name: response
dtype: string
- name: helpfulness
dtype: int32
- name: correctness
dtype: int32
- name: coherence
dtype: int32
- name: complexity
dtype: int32
- name: verbosity
dtype: int32
splits:
- name: train
num_examples: 35331
- name: validation
num_examples: 1789
---

# HelpSteer: Helpfulness SteerLM Dataset

HelpSteer is an open-source Helpfulness Dataset (CC-BY-4.0) that supports aligning models to become more helpful, factually correct and coherent, while being adjustable in terms of the complexity and verbosity of its responses.

Leveraging this dataset and SteerLM, we train a Llama 2 70B to reach **7.54** on MT Bench, the highest among models trained on open-source datasets based on [MT Bench Leaderboard](https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboard) as of 15 Nov 2023.
This model is available on HF at [Llama2-70B-SteerLM-Chat](https://huggingface.co/nvidia/Llama2-70B-SteerLM-Chat).

Try this model instantly for free hosted by us at [NVIDIA AI Playground](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-foundation/models/llama2-70b-steerlm). You can use this in the provided UI or through a limited access API (up to 10, 000 requests within 30 days).
If you would need more requests, we demonstrate how you can set up an inference server at [Llama2-70B-SteerLM-Chat model page on HF](https://huggingface.co/nvidia/Llama2-70B-SteerLM-Chat)

You can also train a model using [NeMo Aligner](https://github.com/NVIDIA/NeMo-Aligner) following [SteerLM training user guide](https://docs.nvidia.com/nemo-framework/user-guide/latest/modelalignment/steerlm.html).

<img src="https://huggingface.co/datasets/nvidia/HelpSteer/resolve/main/mtbench_categories.png" alt="MT Bench Categories" width="800" style="margin-left:'auto' margin-right:'auto' display:'block'"/>

HelpSteer Paper : [HelpSteer: Multi-attribute Helpfulness Dataset for SteerLM](http://arxiv.org/abs/2311.09528)

SteerLM Paper: [SteerLM: Attribute Conditioned SFT as an (User-Steerable) Alternative to RLHF](https://arxiv.org/abs/2310.05344)

Note: [HelpSteer2](https://huggingface.co/datasets/nvidia/HelpSteer2) is the follow up to this dataset and we recommend using HelpSteer2 instead of this dataset.

## Dataset Description

HelpSteer contains 37, 120 samples, each containing a prompt, a response as well as five human-annotated attributes of the response, each ranging between 0 and 4 where higher means better for each attribute.

These attributes are:

1. **Helpfulness**: Overall helpfulness of the response to the prompt.
2. **Correctness**: Inclusion of all pertinent facts without errors.
3. **Coherence**: Consistency and clarity of expression.
4. **Complexity**: Intellectual depth required to write response (i.e. whether the response can be written by anyone with basic language competency or requires deep domain expertise).
5. **Verbosity**: Amount of detail included in the response, relative to what is asked for in the prompt.

## Example

```json
{
"prompt": "What are the three most important things to consider when deciding what technology to use to build an assist device to help an elderly person with basic needs?",
"response": "To build an assistive device to help an elderly person with basic needs, one must consider three crucial things: safety, compatibility, and ease of use. Safety is paramount, as the device must not cause harm to the user. Compatibility with the user's environment and other devices is also essential. Finally, the device must be simple enough for the elderly person to operate.",
"helpfulness": 3,
"correctness": 4,
"coherence": 4,
"complexity": 2,
"verbosity": 2
}
```

## Using the Huggingface Datasets

```python
from datasets import load_dataset

ds = load_dataset("nvidia/HelpSteer")

train = ds['train'] # len(train) = 35331 (95%)
val = ds['validation'] # len(val) = 1789 (5%)
```

## Source

1. Prompts are collected based on a mixture of template-generated (mainly for prompt involving long reference text) and human generated by Scale AI. These prompts relate to the tasks of Rewrite, Summarization, Classification, Extraction, Closed Question Answering, Open Question Answering, Generation and Brainstorming.
2. Responses are generated by an early version of an inhouse LLM. We generate up to 4 responses per prompts using sample techniques to give diverse yet reasonable responses.
3. Annotations of various attributes were done by Scale AI. Annotators rated each response on a Likert 5 scale (between 0 and 4) for each attribute (helpfulness, correctness, coherence, complexity and verbosity).

## Annotation methodology (short)

1. We engaged a select group of contractors via Scale AI. These contractors were provided with comprehensive guidelines that defined each attribute and the criteria for every rating level, together with some annotated examples. These guidelines and examples are detailed in the Appendix of the accompanying paper.
2. The annotation process involved approximately 200 U.S.-based human annotators. Candidates first underwent preliminary assignments, including assessments of English proficiency, to determine eligibility for working on the project. Subsequently, they participated in an introductory training course on the task which ended with a test that involved annotating 35 sample responses. This process ensured not only a thorough understanding of the task requirements but also the delivery of high-quality annotations.
3. Post-annotations, Scale AI performed extensive quality assurance, with each annotation reaching a minimum of two human reviews in addition to automated checks. After receiving the annotations from Scale AI, we conducted our independent quality assurance to make sure that the quality of the annotations was up to our expectations. As a result, some annotations were filtered away to retain only 37, 120 samples.

## Ethical statement
Annotators for the dataset were contracted through Scale AI. Scale AI engages the Anker Methodology, GISC Impact Sourcing Standard, and UN Sustainable Development Goals to provide a fair and competitive pay. The specific pay is calculated based on many factors, including the specific project, the specialized skillset and expertise required, regional costs of living and then transparently listed on Scale AI platform. Scale AI also provides multiple channels for questions and support, including 24/7 support teams, community discussion channels with specially trained moderators, and a “speak up” hotline where contractors can report concerns anonymously. Worker concerns can be submitted to and are reviewed by our Remotasks support team, and pay disputes are reviewed by support specialists trained in this area.

## Citation

If you find this dataset useful, please cite the following works

```bibtex
@misc{wang2023helpsteer,
title={HelpSteer: Multi-attribute Helpfulness Dataset for SteerLM},
author={Zhilin Wang and Yi Dong and Jiaqi Zeng and Virginia Adams and Makesh Narsimhan Sreedhar and Daniel Egert and Olivier Delalleau and Jane Polak Scowcroft and Neel Kant and Aidan Swope and Oleksii Kuchaiev},
year={2023},
eprint={2311.09528},
archivePrefix={arXiv},
primaryClass={cs.CL}
}
```

```bibtex
@misc{dong2023steerlm,
title={SteerLM: Attribute Conditioned SFT as an (User-Steerable) Alternative to RLHF},
author={Yi Dong and Zhilin Wang and Makesh Narsimhan Sreedhar and Xianchao Wu and Oleksii Kuchaiev},
year={2023},
eprint={2310.05344},
archivePrefix={arXiv},
primaryClass={cs.CL}
}
```

---
license: 知识共享署名4.0（CC-BY-4.0）
language:
- 英语（en）
pretty_name: 有用性SteerLM数据集（Helpfulness SteerLM Dataset）
size_categories:
- 10K<n<100K
tags:
- 人类反馈（human-feedback）
dataset_info:
features:
- name: 提示（prompt）
dtype: string
- name: 回复（response）
dtype: string
- name: 有用性（helpfulness）
dtype: int32
- name: 正确性（correctness）
dtype: int32
- name: 连贯性（coherence）
dtype: int32
- name: 复杂度（complexity）
dtype: int32
- name: 详细程度（verbosity）
dtype: int32
splits:
- name: 训练集
num_examples: 35331
- name: 验证集
num_examples: 1789
---

# HelpSteer：有用性SteerLM数据集

HelpSteer 是一个开源的有用性数据集（CC-BY-4.0许可），可用于对齐大语言模型（Large Language Model, LLM），使其更具有用性、事实准确性与连贯性，同时可根据回复的复杂度与详细程度进行调整。

借助该数据集与 SteerLM，我们训练了 Llama 2 70B 模型，在 MT Bench 评测中取得了 7.54 分，截至2023年11月15日，在基于开源数据集训练的模型中位列榜首，相关结果可参见 [MT Bench 排行榜](https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboard)。该模型已在 Hugging Face 平台发布，地址为 [Llama2-70B-SteerLM-Chat](https://huggingface.co/nvidia/Llama2-70B-SteerLM-Chat)。

您可通过我们提供的 [NVIDIA AI Playground](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-foundation/models/llama2-70b-steerlm) 免费即时试用该模型。您可通过内置的用户界面进行体验，或通过受限访问 API 调用（30天内最多可发起10000次请求）。若您需要更多请求次数，可参考 Hugging Face 上的 [Llama2-70B-SteerLM-Chat 模型页面](https://huggingface.co/nvidia/Llama2-70B-SteerLM-Chat) 中的说明搭建推理服务器。

您也可借助 [NeMo Aligner](https://github.com/NVIDIA/NeMo-Aligner) 并遵循 [SteerLM 训练用户指南](https://docs.nvidia.com/nemo-framework/user-guide/latest/modelalignment/steerlm.html) 训练模型。

，宽度800像素，居中显示

HelpSteer 论文：[HelpSteer: Multi-attribute Helpfulness Dataset for SteerLM](http://arxiv.org/abs/2311.09528)

SteerLM 论文：[SteerLM: Attribute Conditioned SFT as an (User-Steerable) Alternative to RLHF](https://arxiv.org/abs/2310.05344)

注：[HelpSteer2](https://huggingface.co/datasets/nvidia/HelpSteer2) 是本数据集的后续版本，我们推荐使用 HelpSteer2 而非本数据集。

## 数据集描述

HelpSteer 共包含37120条样本，每条样本均包含一条提示、一条回复，以及针对该回复的五项人类标注属性，每项属性的评分范围均为0至4，分值越高代表表现越好。

这些属性分别为：
1. **有用性（helpfulness）**：回复对提示的整体有用程度。
2. **正确性（correctness）**：回复包含所有相关事实且无错误。
3. **连贯性（coherence）**：表达的一致性与清晰度。
4. **复杂度（complexity）**：撰写回复所需的知识深度（即该回复是仅具备基础语言能力者即可完成，还是需要深厚的领域专业知识）。
5. **详细程度（verbosity）**：回复中包含的细节量，与提示要求的详细程度相对比。

## 示例

json
{
"prompt": "在决定使用何种技术构建可帮助老年人满足基本需求的辅助设备时，需要考虑的三个最重要的因素是什么？",
"response": "要构建帮助老年人满足基本需求的辅助设备，需考虑三个关键因素：安全性、兼容性与易用性。安全性至关重要，因为设备不得对用户造成伤害。设备需与用户的使用环境及其他设备兼容，这一点同样必要。最后，设备必须足够简单，便于老年人操作。",
"helpfulness": 3,
"correctness": 4,
"coherence": 4,
"complexity": 2,
"verbosity": 2
}

## 使用 Hugging Face 数据集

python
from datasets import load_dataset

ds = load_dataset("nvidia/HelpSteer")

train = ds['train'] # len(train) = 35331（占比95%）
val = ds['validation'] # len(val) = 1789（占比5%）

## 数据来源

1. 提示语混合了模板生成（主要用于涉及长参考文本的提示）与 Scale AI 团队人工生成的内容。这些提示涵盖了改写、摘要、分类、信息提取、封闭域问答、开放域问答、内容生成与头脑风暴等任务类型。
2. 回复由早期版本的自研大语言模型生成。我们通过采样技术为每条提示生成最多4条回复，以获得多样化且合理的回复结果。
3. 各项属性的标注工作由 Scale AI 团队完成。标注人员针对每条回复的五项属性（有用性、正确性、连贯性、复杂度与详细程度），采用李克特5分量表（0至4分）进行评分。

## 标注方法（精简版）

1. 我们通过 Scale AI 平台签约了一批精选的标注人员。为标注人员提供了详尽的指南，明确了每项属性的定义与各评分等级的标准，并附带了部分标注示例。该指南与示例详见配套论文的附录部分。
2. 标注过程共有约200名美国本土的人工标注人员参与。候选人员首先需完成初步任务，包括英语能力评估，以确定其是否具备参与本项目的资格。随后，他们需参加针对该任务的入门培训课程，课程结束后需完成35条回复的标注测试。该流程可确保标注人员充分理解任务要求，同时保障标注质量。
3. 标注完成后，Scale AI 团队开展了全面的质量管控：每条标注除需通过自动化检查外，还至少需经过两名人工审核。在收到 Scale AI 交付的标注结果后，我们也进行了独立的质量核查，以确保标注质量符合我们的预期。最终我们筛选保留了37120条有效样本。

## 伦理声明

标注人员均通过 Scale AI 平台签约。Scale AI 采用 Anker 方法论、GISC 影响采购标准与联合国可持续发展目标，为标注人员提供公平且具有竞争力的薪酬。具体薪酬基于多个因素计算，包括项目类型、所需的专业技能与知识、所在地区的生活成本，并在 Scale AI 平台上透明公示。Scale AI 还提供了多渠道的咨询与支持服务，包括全天候支持团队、配有专业版主的社区讨论频道，以及标注人员可匿名提交诉求的“发声”热线。标注人员的诉求将由我们的 Remotasks 支持团队接收并审核，薪酬纠纷将由相关领域的支持专员处理。

## 引用

若您认为本数据集对您的工作有所帮助，请引用以下文献：

bibtex
@misc{wang2023helpsteer,
title={HelpSteer: Multi-attribute Helpfulness Dataset for SteerLM},
author={Zhilin Wang and Yi Dong and Jiaqi Zeng and Virginia Adams and Makesh Narsimhan Sreedhar and Daniel Egert and Olivier Delalleau and Jane Polak Scowcroft and Neel Kant and Aidan Swope and Oleksii Kuchaiev},
year={2023},
eprint={2311.09528},
archivePrefix={arXiv},
primaryClass={cs.CL}
}

bibtex
@misc{dong2023steerlm,
title={SteerLM: Attribute Conditioned SFT as an (User-Steerable) Alternative to RLHF},
author={Yi Dong and Zhilin Wang and Makesh Narsimhan Sreedhar and Xianchao Wu and Oleksii Kuchaiev},
year={2023},
eprint={2310.05344},
archivePrefix={arXiv},
primaryClass={cs.CL}
}