PKU-SafeRLHF-30K

该数据集包含约27,000个训练条目和3,000个测试条目。每个条目包含一个元组（x, y0, y1），并附有注释，指明哪个回应更有帮助，哪个更安全，以及对每个回应的二元安全指示器。此外，该数据集被用于评估SafeDPO算法在减少不安全回应和提升回应有用性方面的性能。该任务旨在训练和测试语言模型在安全对齐方面的算法。

This dataset contains approximately 27,000 training instances and 3,000 test instances. Each instance comprises a tuple (x, y0, y1), accompanied by annotations that indicate which of the two responses is more helpful, which is safer, as well as a binary safety indicator for each response. Furthermore, this dataset is utilized to evaluate the performance of the SafeDPO algorithm in reducing unsafe responses and enhancing the helpfulness of model responses. This task aims to train and test algorithms for the safety alignment of language models.