LEGEND

LEGEND数据集是由四川大学和北京人工智能研究院共同开发的，专注于增强偏好数据集的边界标注，以提高奖励模型在区分微妙安全差异方面的性能。该数据集通过利用表示工程在大型语言模型（LLM）的嵌入空间内构建特定的安全方向，从而自动标注偏好边界。LEGEND数据集的应用领域主要集中在提升LLM的安全对话能力，解决奖励模型在安全对齐中的精确度问题。创建过程中，LEGEND通过发现安全向量和边界标注两个步骤，利用LLM生成的有害和无害响应的嵌入差异来构建标准边界向量（SMV），进而通过SMV测量配对响应之间的安全距离，实现边界标注。

The LEGEND dataset was co-developed by Sichuan University and Beijing Institute for AI, focusing on enhancing boundary annotation for preference datasets to improve the performance of reward models in distinguishing subtle safety discrepancies. This dataset constructs specific safety orientations within the embedding space of Large Language Models (LLMs) via representation engineering, thereby enabling automatic annotation of preference boundaries. The application scenarios of the LEGEND dataset primarily focus on enhancing the safety dialogue capabilities of LLMs and addressing the accuracy challenges of reward models in safety alignment. During its development, LEGEND follows two core steps: safety vector discovery and boundary annotation. It first constructs Standard Margin Vector (SMV) by leveraging the embedding differences between harmful and harmless responses generated by LLMs, then measures the safety distance between paired responses using SMV to complete the boundary annotation.