MMTE: Corpus and Metrics for Evaluating Machine Translation Quality of Metaphorical Language

We introduce a <b>multilingual parallel corpus</b> specifically curated to evaluate the <b>figurative quality</b> of machine translation (MT) outputs. The dataset consists of English sentences containing metaphors, along with their corresponding <b>human post-edited translations</b> in multiple target languages, including <b>Chinese</b> and <b>Italian</b>.The corpus was constructed through a two-stage process:<b>Initial Translation</b>: Sentences containing figurative expressions were translated using standard MT systems (e.g., LLM-based or neural MT).<b>Human Post-editing</b>: These translations were subsequently <b>post-edited by native speakers or professional translators</b> with linguistics training to improve fluency, semantic accuracy, and especially fidelity to the figurative meaning.Each example in the dataset includes:The <b>original English sentence</b> with the figurative expression.The <b>raw machine translation</b> in the target language.The <b>post-edited version</b> of the translation.<b>Annotations</b> based on four human evaluation metrics:<b>Quality</b>: Considering fluency, intelligibility, fidelity, and overall quality.<b>Metaphorical Equivalence</b>: How well the metaphorical meaning is preserved.<b>Emotion</b>: Whether the emotional tone or affect is maintained.<b>Authenticity</b>: Whether the translation sounds natural and idiomatic in the target language.Each sample is <b>triple-annotated</b>, with disagreements resolved through professional translator review, ensuring high inter-annotator reliability.This dataset supports research in:Evaluating figurative language in MT.Improving translation systems’ handling of metaphors.Developing automatic metrics that align better with human judgements of figurative quality.