hh-rlhf-helpful-base

# HH-RLHF-Helpful-Base Dataset

## Summary

The HH-RLHF-Helpful-Base dataset is a processed version of [Anthropic's HH-RLHF](https://huggingface.co/datasets/Anthropic/hh-rlhf) dataset, specifically curated to train models using the [TRL library](https://github.com/huggingface/trl) for preference learning and alignment tasks. It contains pairs of text samples, each labeled as either "chosen" or "rejected," based on human preferences regarding the helpfulness of the responses. This dataset enables models to learn human preferences in generating helpful responses, enhancing their ability to assist users effectively.

## Data Structure

- **Format**: [Conversational](https://huggingface.co/docs/trl/main/dataset_formats#conversational)
- **Type**: [Preference](https://huggingface.co/docs/trl/main/dataset_formats#preference)

Columns:
- `"pompt"`: The user query.
- `"chosen"`: A response deemed helpful by human evaluators.
- `"rejected"`: A response considered less helpful or unhelpful.

This structure allows models to learn to prefer the _chosen_ response over the _rejected_ one, thereby aligning with human preferences in helpfulness.

## Generation script

The script used to generate this dataset can be found [here](https://github.com/huggingface/trl/blob/main/examples/datasets/hh-rlhf-helpful-base.py).

# HH-RLHF-Helpful-Base 数据集

## 摘要
HH-RLHF-Helpful-Base 数据集是[Anthropic的HH-RLHF数据集](https://huggingface.co/datasets/Anthropic/hh-rlhf)的处理版本，专为使用[TRL库（TRL library）](https://github.com/huggingface/trl)训练模型以开展偏好学习与对齐任务而精心构建。该数据集包含成对的文本样本，每对样本均基于人类对回复有用性的偏好，被标记为"已选（chosen）"或"未选（rejected）"。此数据集可使模型学习生成有用回复时的人类偏好，进而提升其有效协助用户的能力。

## 数据结构
- **格式**：对话式（Conversational）
- **类型**：偏好式（Preference）

字段说明：
- `"prompt"`：用户查询内容（原文笔误为pompt）。
- `"chosen"`：经人类评估者认定为有用的回复。
- `"rejected"`：被认为有用性较低或完全无用的回复。

该数据结构可使模型学习优先选择已选回复而非未选回复，从而在回复有用性层面对齐人类偏好。

## 生成脚本
用于生成该数据集的脚本可在[此处](https://github.com/huggingface/trl/blob/main/examples/datasets/hh-rlhf-helpful-base.py)获取。