IndiBias

IndiBias是一个专为评估印度社会文化背景下大型语言模型中的社会偏见而设计的综合基准数据集。该数据集由印度理工学院孟买分校创建，包含800对句子，用于测量不同人口统计学中的偏见。数据集通过过滤和翻译现有的CrowS-Pairs数据集，并利用ChatGPT和InstructGPT增强，涵盖了性别、宗教、种姓、年龄、地区、外貌和职业等七个偏见维度。此外，数据集还构建了一个资源，以解决三个交叉维度的交叉偏见。IndiBias旨在通过提供一个可靠的数据集，帮助研究人员更好地理解和减少语言模型中的偏见，特别是在印度这样的多元化社会中。

IndiBias is a comprehensive benchmark dataset specifically designed to evaluate social biases in large language models within India's socio-cultural context. Developed by the Indian Institute of Technology Bombay, it contains 800 sentence pairs for measuring biases across diverse demographic groups. The dataset is constructed by filtering and translating the existing CrowS-Pairs dataset, and augmented with the assistance of ChatGPT and InstructGPT, covering seven bias dimensions including gender, religion, caste, age, region, physical appearance, and occupation. Additionally, the dataset also provides resources to address intersectional biases across three intersecting demographic dimensions. IndiBias aims to help researchers better understand and mitigate biases in language models, particularly in diverse societies like India, by offering a reliable benchmark dataset.