The Usability of Neural Machine Translation in Creative-Text Post-Editing: Evidence from Users’ Performance and Perception

Recent developments in translation technologies can help neural machine translation (NMT) to generate high-quality translation. This study aims to investigate the usability of NMT in creative-text post-editing with evidence from users' performance and perception. Based on the concept of usability and prior research, we developed an NMT usability assessment framework including three dimensions – efficiency, effectiveness, and satisfaction. By analysing data of three dimensions collected from key logging, screen recording, questionnaires, and retrospective interviews, we found that using NMT in creative-text post-editing (or MTPE) was significantly more efficient and yielded higher acceptability than human translation. Most participants hold a positive yet cautious attitude toward the usability of NMT. It suggests that MTPE as a type of human-computer interaction shows excellent potential in producing a high-quality creative-text translation efficiently. However, the study also reveals the limitations of using NMT in translating creative texts and raises both ethical and legal concerns regarding plagiarism.