ibm-research/SocialStigmaQA-JA

SocialStigmaQA-JA数据集是一个用于测试大型语言模型社会偏见的日语版本数据集，包含了大约10K个经过精心翻译的提示，旨在系统地测试社会偏见和模型的鲁棒性。该数据集从93个美国中心主义偏见列表和手 curated问答模板构建而成，这些模板涉及社会情境。数据集特别关注每个模板的两个方面：带有偏见的答案选择和提示风格。

SocialStigmaQA-JA is a Japanese version of the SocialStigmaQA dataset designed to test the social bias of large language models. It contains approximately 10K prompts carefully translated from the original to systematically test for both social bias and model robustness. The dataset is constructed from a documented list of 93 US-centric stigmas and hand-curated QA templates involving social situations. An important aspect of the dataset is the attention to two aspects of each template - biased answer choices and prompt styles.