Grep-BiasIR

Grep-BiasIR 是一个全新的经过彻底审计的数据集，旨在促进对 IR 系统检索结果中的性别偏见的研究。信息检索 (IR) 系统提供的内容可以反映现有的社会偏见和刻板印象。检索结果中的这种偏差会导致在社会和系统中进一步建立和加强刻板印象。为了便于研究 IR 系统检索结果中的性别偏见，我们引入了用于信息检索的性别表示-偏见 (Grep-BiasIR)，这是一个全新的、经过彻底审计的数据集，由 118 个对偏见敏感的中性搜索查询组成。这组查询涵盖了广泛的与性别相关的主题，对于这些主题，搜索结果中对性别的偏见表示可以被认为是社会问题。每个查询都附有一份相关文档和一份非相关文档，其中该文档还提供了女性、男性和中立的三种变体。该数据集可在 https://github.com/KlaraKrieg/GrepBiasIR 获得。

Grep-BiasIR is a novel, thoroughly audited dataset designed to advance research on gender bias in the retrieval results of Information Retrieval (IR) systems. The content delivered by IR systems can reflect existing societal biases and stereotypes. Such biases in retrieval outcomes can further entrench and reinforce stereotypes across both society and IR systems themselves. To facilitate research on gender bias in IR system retrieval results, we introduce Gender Representation-Bias for Information Retrieval (Grep-BiasIR) — a novel, thoroughly audited dataset consisting of 118 bias-aware neutral search queries. This collection of queries covers a broad range of gender-related topics, where biased gender representations in search results constitute a social concern. Each query is paired with one relevant document and one non-relevant document, each of which features three variants: female, male, and neutral. This dataset is publicly available at https://github.com/KlaraKrieg/GrepBiasIR.

搜集汇总

背景与挑战

背景概述

Grep-BiasIR是一个专门用于研究信息检索系统性别偏见的数据集，包含118个中性搜索查询及其相关文档的三种性别变体。该数据集由学术机构于2022年发布，旨在促进对检索结果中性别偏见表示的研究。