heqianwan/ultrafeedback_binarized

这是[UltraFeedback数据集](https://huggingface.co/datasets/openbmb/UltraFeedback)的预处理版本，用于训练[Zephyr-7Β-β](https://huggingface.co/HuggingFaceH4/zephyr-7b-beta)，这是一个在7B参数规模上最先进的聊天模型。原始的UltraFeedback数据集包含64k个提示，每个提示附带来自各种开放和专有模型的四个完成。GPT-4用于根据帮助性和诚实性等标准为每个完成分配分数。为了创建`UltraFeedback Binarized`，我们选择最高的`overall_score`作为“选定”完成，并随机选择其余三个中的一个作为“拒绝”完成。这定义了用于奖励建模或DPO等技术的偏好建模分割。我们还创建了用于监督微调（SFT）的分割，使用“选定”列作为建模的对话，以及涉及生成的分割，如拒绝采样或PPO。有关数据集处理的详细信息，请参阅随附的[脚本](https://huggingface.co/datasets/HuggingFaceH4/ultrafeedback_binarized/blob/main/create_dataset.py)。

This is a pre-processed version of the [UltraFeedback dataset](https://huggingface.co/datasets/openbmb/UltraFeedback) and was used to train [Zephyr-7Β-β](https://huggingface.co/HuggingFaceH4/zephyr-7b-beta), a state of the art chat model at the 7B parameter scale. The original UltraFeedback dataset consists of 64k prompts, where each prompt is accompanied with four model completions from a wide variety of open and proprietary models. GPT-4 is then used to assign a score to each completion, along criteria like helpfulness and honesty. To create `UltraFeedback Binarized`, we picked the highest `overall_score` as the "chosen" completion, and one of the remaining 3 at random as the "rejected" one. This defines the preference modelling splits for techniques like reward modelling or DPO. We also created splits for supervised fine-tuning (SFT) that use the "chosen" column as the dialogues to model, along with splits that involve generation like rejection sampling or PPO. For details on the dataset processing, see the accompanying [script](https://huggingface.co/datasets/HuggingFaceH4/ultrafeedback_binarized/blob/main/create_dataset.py).