100PoisonMpts: 中文大模型治理数据集

业内首个大语言模型治理开源中文数据集，十多位知名专家学者作为首批“给AI的100瓶毒药”的标注工程师，提出100个诱导偏见、歧视回答的刁钻问题，并对大模型的回答进行标注，完成与AI从“投毒”和“解毒”的攻防。

This is the industry's first open-source Chinese dataset dedicated to large language model (LLM) governance. Over a dozen renowned experts and scholars served as the first batch of annotators for the "100 Bottles of Poisons for AI" initiative. They proposed 100 tricky questions designed to elicit biased and discriminatory responses from LLMs, annotated the outputs generated by the models, and completed the offensive and defensive practice between human annotators and AI centered on "prompt poisoning" and "prompt detoxification".

搜集汇总

背景与挑战

背景概述

100PoisonMpts是一个中文大模型治理数据集，由阿里巴巴团队联合多位领域专家构建，旨在通过专家标注的诱导性问题和答案，评估和提升大语言模型在反歧视、同理心等方面的安全性和价值观对齐。数据集包含906条样本，覆盖法理学、心理学、环境科学等多个专业领域，采用JSON格式，可用于大模型的对齐研究和优化。