crows-pairs-multilingual

This is the Github repo for French CrowS-Pairs, a challenge dataset for measuring the degree to which French stereotypical biases present in the masked language models (MLMs). This challenge dataset builds on the Crows-Pairs corpus (Nangia et al. 2020) as follows: 1/ it addresses issues reported by Blodgett et al. (2021) through a revision of the original content in English 2/ it provides a translation into French of revised Crows-Pairs content and 3/ it provides newly collected stereotypes in native French using the original collection methodology described by Nangia et al. (2020).