UnStereoEval (USE)
收藏arXiv2025-09-30 收录
下载链接：
https://ucinlp.github.io/unstereo-eval
下载链接
链接失效反馈官方服务：
资源简介：
该数据集采用了一种新颖的框架，专门用于在无刻板印象的场景中研究性别偏见问题。该框架利用基于预训练数据统计的句子级评分。此外，数据集还包括三个基准测试：USE-5、USE-10和USE-20，这些基准测试是通过使用5、10和20个单词长度的句子生成的。该研究涵盖了28个测试模型，旨在评估语言模型在无刻板印象场景中的公平性。
This dataset adopts a novel framework specifically designed for investigating gender bias in stereotype-free scenarios. This framework utilizes sentence-level scores based on statistical analyses of pre-training data. Additionally, the dataset includes three benchmark tests: USE-5, USE-10, and USE-20, which are generated using sentences with lengths of 5, 10, and 20 words respectively. The research encompasses 28 test models, aiming to evaluate the fairness of language models in stereotype-free scenarios.
提供机构：
UCI NLP
