ACES and Span-ACES

ACES consists of 36,476 examples covering 146 language pairs and representing challenges from 68 phenomena for evaluating machine translation metrics. We focus on translation accuracy errors and base the phenomena covered in our challenge set on the Multidimensional Quality Metrics (MQM) ontology. The phenomena range from simple perturbations at the word/character level to more complex errors based on discourse and real-world knowledge. 29.01.2024: We also release Span-ACES, which is an extension to the ACES dataset. The errors in incorrect-translation are explicitly marked in a span format.