PAT-TQE-2026-1 Evaluation Dashboard and Reports

This repository is a dashboard that tracks translation quality evaluations and inter-annotator agreement for the PAT-TQE-2026-1 project series. This effort has two main objectives: first, to train a team of professional evaluators to produce consistent, deployable annotations of translation quality; and second, to use the annotations to draw conclusions about corpus-informed, document-level automatic translation quality produced from our Pragmatic Auto-Translator (PAT). The reports shared here have been generated from Label Studio annotations of translation quality errors following a customized MQM typology. The reports were generated using the notebooks made available here: https://github.com/locessentials/translation-quality-evaluation/ You can learn about our findings and the broader initial context for this research in our forthcoming paper, "Can an Old Dog Be Taught New Tricks? Taking LLMs Beyond Sentence Level Automatic Translation", which will be published in the proceedings of HCI International 2026. This repository on GitHub: https://github.com/locessentials/tqe-2026-1 The dashboard hosted on the web by GitHub pages: https://locessentials.github.io/tqe-2026-1/