HelpSteer2 人类偏好对齐数据集

HelpSteer2 是一个由英伟达和 Scale AI 于 2024 年合作创建的开源数据集，旨在训练能够指导大型语言模型 (LLMs) 生成符合人类偏好的高质量回答的奖励模型，相关论文成果为「HelpSteer2: Open-source dataset for training top-performing reward models」。它是在 HelpSteer 数据集的基础上进行的更新，以适应当前更强大的 LLMs 。 HelpSteer2 包含约一万对回答，尽管数量上比现有的偏好数据集少一个数量级，但它在训练奖励模型方面非常高效。

HelpSteer2 is an open-source dataset jointly developed by NVIDIA and Scale AI in 2024, targeted at training reward models that steer large language models (LLMs) to produce high-quality responses aligned with human preferences. The accompanying scholarly paper is titled "HelpSteer2: Open-source dataset for training top-performing reward models". As an updated iteration built upon the original HelpSteer dataset, it is tailored for the more advanced contemporary LLMs. HelpSteer2 encompasses roughly 10,000 response pairs. Though its scale is an order of magnitude smaller than prevailing preference datasets, it demonstrates remarkable efficiency in reward model training.

搜集汇总

背景与挑战

背景概述

HelpSteer2是由英伟达和Scale AI合作创建的开源人类偏好对齐数据集，旨在训练奖励模型以指导大型语言模型生成符合人类偏好的高质量回答。该数据集包含约一万对回答，在训练奖励模型方面表现出高效性，例如训练的Llama 3 70B模型在Reward-Bench上取得了92.0%的得分。