FBK-MT/mGeNTE-supplementary

---
license: apache-2.0
language:
- en
- es
- it
- de
---

Supplementary data released alongside the paper: **[Mind the Inclusivity Gap: Multilingual Gender-Neutral Translation Evaluation with mGeNTE](https://arxiv.org/html/2501.09409v3#:~:text=To%20address%20this%20gap%2C%20we%20introduce%20mGeNTE%2C%20an,inclusive%20translation%20with%20state-of-the-art%20instruction-following%20language%20models%20%28LMs%29.)**.

The official mGeNTE dataset is at [this link](https://huggingface.co/datasets/FBK-MT/mGeNTE).

## Structure

The supplementary material is organized as follows:

- `attributions`: contains all the token-level attributions to be used with the class defined in our main repository, found at [this link](https://github.com/g8a9/mgente-gap). Moreover, we also include a "processed_" version of each file where we aggregated the contributions following the context parts as described in the paper.
- `gnt`: contains the gender neutral evaluation labels for translations as they are estimated by an LLM judge as described in the paper.
- `manual_eval`: contains TSV files manually annotated for gender neutral evaluation labels of the translation, which, as described in the paper, are used to estimate the accuracy of the LLM judge evaluator. Manual evaluation guidelines available at [this link](https://github.com/g8a9/mgente-gap/tree/main/guidelines)
- `translations`: contains all the raw translations as they are generated by the models described in the paper.

## Contacts

If you have any doubts or curiosity about this material, feel free to open an issue on this repository or [the official one](https://github.com/g8a9/mgente-gap).

## Citation

If you use any of the materials related to the paper, please cite:

```bibtex
@misc{savoldi2025mindinclusivitygapmultilingual,
title={Mind the Inclusivity Gap: Multilingual Gender-Neutral Translation Evaluation with mGeNTE},
author={Beatrice Savoldi and Giuseppe Attanasio and Eleonora Cupin and Eleni Gkovedarou and Janiça Hackenbuchner and Anne Lauscher and Matteo Negri and Andrea Piergentili and Manjinder Thind and Luisa Bentivogli},
year={2025},
eprint={2501.09409},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2501.09409},
}
```