Columbia-NLP/DPO-PKU-SafeRLHF

我们重新格式化了[PKU-Alignment/PKU-SafeRLHF](https://huggingface.co/datasets/PKU-Alignment/PKU-SafeRLHF)数据集，以适应本集合中所有DPO数据集的通用格式。具体来说，我们：1. 只保留标记为安全的响应；2. 由于原始数据集中没有标记分数，我们使用`chosen=10.0`和`rejected=1.0`作为占位符。

The DPO-PKU-SafeRLHF dataset is reformatted from the PKU-Alignment/PKU-SafeRLHF dataset, used to support the training of the LION series models. It includes features such as prompt, prompt_id, chosen and rejected content, each containing content and role. Additionally, it includes messages, scores for chosen and rejected (score_chosen and score_rejected), and other information such as safety flags for chosen and rejected content (is_chosen_safe and is_rejected_safe) and source. The dataset is split into training and testing parts, containing 122274 and 14047 samples respectively. The purpose of the dataset is to ensure the reproducibility and performance improvement of models, especially when using publicly available datasets.