GroNLP/ik-nlp-22_pestyle

--- annotations_creators: - machine-generated - expert-generated language_creators: - found language: - en - it license: - other multilinguality: - translation size_categories: - 1K<n<10K source_datasets: - original task_categories: - translation pretty_name: iknlp22-pestyle --- # Dataset Card for IK-NLP-22 Project 1: A Study in Post-Editing Stylometry ## Table of Contents - [Dataset Card for IK-NLP-22 Project 1: A Study in Post-Editing Stylometry](#dataset-card-for-ik-nlp-22-project-1-a-study-in-post-editing-stylometry) - [Table of Contents](#table-of-contents) - [Dataset Description](#dataset-description) - [Dataset Summary](#dataset-summary) - [Languages](#languages) - [Dataset Structure](#dataset-structure) - [Data Instances](#data-instances) - [Data Fields](#data-fields) - [Data Splits](#data-splits) - [Train Split](#train-split) - [Test splits](#test-splits) - [Dataset Creation](#dataset-creation) - [Additional Information](#additional-information) - [Dataset Curators](#dataset-curators) - [Licensing Information](#licensing-information) - [Citation Information](#citation-information) ## Dataset Description - **Source:** [FLORES-101](https://huggingface.co/datasets/gsarti/flores_101) - **Point of Contact:** [Gabriele Sarti](mailto:[e-mail]) ### Dataset Summary This dataset contains a sample of sentences taken from the [FLORES-101](https://huggingface.co/datasets/gsarti/flores_101) dataset that were either translated from scratch or post-edited from an existing automatic translation by three human translators. Translation were performed for the English-Italian language pair, and translators' behavioral data (keystrokes, pauses, editing times) were collected using the [PET](https://github.com/wilkeraziz/PET) platform. This dataset is made available for final projects of the 2022 edition of the Natural Language Processing course at the [Information Science Master's Degree](https://www.rug.nl/masters/information-science/?lang=en) at the University of Groningen, taught by [Arianna Bisazza](https://research.rug.nl/en/persons/arianna-bisazza) and [Gabriele Sarti](https://research.rug.nl/en/persons/gabriele-sarti) with the assistance of [Anjali Nair](https://nl.linkedin.com/in/anjalinair012). **Disclaimer**: *This repository is provided without direct data access due to currently unpublished results.* _**For this reason, it is strictly forbidden to share or publish all the data associated to this repository**_. *Students will be provided with a compressed folder containing the data upon choosing a project based on this dataset. To load the dataset using 🤗 Datasets, download and unzip the provided folder and pass it to the* `load_dataset` *method as:* `datasets.load_dataset('GroNLP/ik-nlp-22_pestyle', 'full', data_dir='path/to/unzipped/folder')` ### Languages The language data of is in English (BCP-47 `en`) and Italian (BCP-47 `it`) ## Dataset Structure ### Data Instances The dataset contains four configurations: `full`, `test_mask_subject`, `test_mask_modality`, `test_mask_time`. `full` contains the main `train` split in which all fields are available. The other three, `test_mask_subject`, `test_mask_modality`, `test_mask_time`, contain a `test` split each with different fields removed to avoid information leaking during evaluation. See more details in the [Data Splits](#data-splits) section. ### Data Fields The following fields are contained in the training set: |Field|Description| |-----|-----------| |`item_id` | The sentence identifier. The first digits of the number represent the document containing the sentence, while the last digit of the number represents the sentence position inside the document. Documents can contain from 3 to 5 semantically-related sentences each. | |`subject_id` | The identifier for the translator performing the translation from scratch or post-editing task. Values: `t1`, `t2` or `t3`. | |`modality` | The modality of the translation task. Values: `ht` (translation from scratch), `pe1` (post-editing Google Translate translations), `pe2` (post-editing [mBART](https://huggingface.co/facebook/mbart-large-50-one-to-many-mmt) translations). | |`src_text` | The original source sentence extracted from Wikinews, wikibooks or wikivoyage. | |`mt_text` | Missing if tasktype is `ht`. Otherwise, contains the automatically-translated sentence before post-editing. | |`tgt_text` | Final sentence produced by the translator (either via translation from scratch of `sl_text` or post-editing `mt_text`) | |`edit_time` | Total editing time for the translation in seconds. | |`k_total` | Total number of keystrokes for the translation. | |`k_letter` | Total number of letter keystrokes for the translation. | |`k_digit` | Total number of digit keystrokes for the translation. | |`k_white` | Total number of whitespace keystrokes for the translation. | |`k_symbol` | Total number of symbol (punctuation, etc.) keystrokes for the translation. | |`k_nav` | Total number of navigation keystrokes (left-right arrows, mouse clicks) for the translation. | |`k_erase` | Total number of erase keystrokes (backspace, cancel) for the translation. | |`k_copy` | Total number of copy (Ctrl + C) actions during the translation. | |`k_cut` | Total number of cut (Ctrl + X) actions during the translation. | |`k_paste` | Total number of paste (Ctrl + V) actions during the translation. | |`n_pause_geq_300` | Number of pauses of 300ms or more during the translation. | |`len_pause_geq_300` | Total duration of pauses of 300ms or more, in milliseconds. | |`n_pause_geq_1000` | Number of pauses of 1s or more during the translation. | |`len_pause_geq_1000` | Total duration of pauses of 1000ms or more, in milliseconds. | |`num_annotations` | Number of times the translator focused the texbox for performing the translation of the sentence during the translation session. E.g. 1 means the translation was performed once and never revised. | |`n_insert` | Number of post-editing insertions (empty for modality `ht`) computed using the [tercom](https://github.com/jhclark/tercom) library. | |`n_delete` | Number of post-editing deletions (empty for modality `ht`) computed using the [tercom](https://github.com/jhclark/tercom) library. | |`n_substitute` | Number of post-editing substitutions (empty for modality `ht`) computed using the [tercom](https://github.com/jhclark/tercom) library. | |`n_shift` | Number of post-editing shifts (empty for modality `ht`) computed using the [tercom](https://github.com/jhclark/tercom) library. | |`bleu` | Sentence-level BLEU score between MT and post-edited fields (empty for modality `ht`) computed using the [SacreBLEU](https://github.com/mjpost/sacrebleu) library with default parameters. | |`chrf` | Sentence-level chrF score between MT and post-edited fields (empty for modality `ht`) computed using the [SacreBLEU](https://github.com/mjpost/sacrebleu) library with default parameters. | |`ter` | Sentence-level TER score between MT and post-edited fields (empty for modality `ht`) computed using the [tercom](https://github.com/jhclark/tercom) library. | |`aligned_edit` | Aligned visual representation of REF (`mt_text`), HYP (`tl_text`) and edit operations (I = Insertion, D = Deletion, S = Substitution) performed on the field. Replace `\\n` with `\n` to show the three aligned rows.| ### Data Splits | config| train| test| |------:|-----:|----:| |`main` | 1170 | 120 | #### Train Split The `train` split contains a total of 1170 triplets (or pairs, when translation from scratch is performed) annotated with behavioral data produced during the translation. The following is an example of the subject `t3` post-editing a machine translation produced by system 2 (tasktype `pe2`) taken from the `train` split. The field `aligned_edit` is showed over three lines to provide a visual understanding of its contents. ```json { "item_id": 1072, "subject_id": "t3", "tasktype": "pe2", "src_text": "At the beginning dress was heavily influenced by the Byzantine culture in the east.", "mt_text": "All'inizio il vestito era fortemente influenzato dalla cultura bizantina dell'est.", "tgt+text": "Inizialmente, l'abbigliamento era fortemente influenzato dalla cultura bizantina orientale.", "edit_time": 45.687, "k_total": 51, "k_letter": 31, "k_digit": 0, "k_white": 2, "k_symbol": 3, "k_nav": 7, "k_erase": 3, "k_copy": 0, "k_cut": 0, "k_paste": 0, "n_pause_geq_300": 9, "len_pause_geq_300": 40032, "n_pause_geq_1000": 5, "len_pause_geq_1000": 38392, "num_annotations": 1, "n_insert": 0.0, "n_delete": 1.0, "n_substitute": 3.0, "n_shift": 0.0, "bleu": 47.99, "chrf": 62.05, "ter": 40.0, "aligned_edit: "REF: all'inizio il vestito era fortemente influenzato dalla cultura bizantina dell'est.\\n HYP: ********** inizialmente, l'abbigliamento era fortemente influenzato dalla cultura bizantina orientale.\\n EVAL: D S S S" } ``` The text is provided as-is, without further preprocessing or tokenization. #### Test splits The three `test` splits (one per configuration) contain the same 120 entries each, following the same structure as `train`. Each test split omit some of the fields to prevent leakage of information: - In `test_mask_subject` the `subject_id` is absent, for the main task of post-editor stylometry. - In `test_mask_modality` the following fields are absent for the modality prediction extra task: `modality`, `mt_text`, `n_insert`, `n_delete`, `n_substitute`, `n_shift`, `ter`, `bleu`, `chrf`, `aligned_edit`. - In `test_mask_time` the following fields are absent for the time and pause prediction extra task: `edit_time`, `n_pause_geq_300`, `len_pause_geq_300`, `n_pause_geq_1000`, and `len_pause_geq_1000`. ### Dataset Creation The dataset was parsed from PET XML files into CSV format using a script adapted from the one by [Antonio Toral](https://research.rug.nl/en/persons/antonio-toral-ruiz) found at the following link: [https://github.com/antot/postediting_novel_frontiers](https://github.com/antot/postediting_novel_frontiers) ## Additional Information ### Dataset Curators For problems related to this 🤗 Datasets version, please contact us at [[e-mail]](mailto:[e-mail]). ### Licensing Information It is forbidden to share or publish the data associated with this 🤗 Dataset version. ### Citation Information No citation information is provided for this dataset.

原始信息汇总

数据集概述

数据集名称

• 名称: iknlp22-pestyle
• 别名: IK-NLP-22 Project 1: A Study in Post-Editing Stylometry

数据集内容

• 语言: 英语 (en) 和意大利语 (it)
• 多语言性: 翻译
• 许可: 其他
• 大小: 1K<n<10K
• 任务类别: 翻译

数据集结构

• 数据实例: 包含四种配置：full, test_mask_subject, test_mask_modality, test_mask_time。
• 数据字段:
  • item_id: 句子标识符
  • subject_id: 翻译者标识符
  • modality: 翻译任务模式
  • src_text: 原始源句子
  • mt_text: 机器翻译句子（如任务类型为ht则缺失）
  • tgt_text: 翻译者最终产出的句子
  • edit_time: 翻译编辑总时间（秒）
  • k_total: 总按键次数
  • k_letter: 字母按键次数
  • k_digit: 数字按键次数
  • k_white: 空格按键次数
  • k_symbol: 符号按键次数
  • k_nav: 导航按键次数
  • k_erase: 删除按键次数
  • k_copy: 复制操作次数
  • k_cut: 剪切操作次数
  • k_paste: 粘贴操作次数
  • n_pause_geq_300: 300ms以上暂停次数
  • len_pause_geq_300: 300ms以上暂停总时长（毫秒）
  • n_pause_geq_1000: 1s以上暂停次数
  • len_pause_geq_1000: 1s以上暂停总时长（毫秒）
  • num_annotations: 翻译者聚焦文本框执行翻译次数
  • n_insert: 插入操作次数
  • n_delete: 删除操作次数
  • n_substitute: 替换操作次数
  • n_shift: 移动操作次数
  • bleu: BLEU评分
  • chrf: chrF评分
  • ter: TER评分
  • aligned_edit: 对齐编辑操作的视觉表示

数据分割

• 训练集: 1170个实例
• 测试集: 120个实例，分为三个子集，每个子集根据任务需求省略特定字段。

数据集创建

• 来源: 从PET XML文件解析并转换为CSV格式。

许可证和使用限制

• 许可证: 其他
• 使用限制: 禁止分享或发布与该数据集相关的数据。

搜集汇总

数据集介绍

构建方式

该数据集源自FLORES-101语料库中的英文句子样本，由三位人类译员通过PET平台完成从零翻译或机器翻译后编辑任务，涵盖英语至意大利语的语言对。翻译过程中系统化采集了译员的击键、停顿及编辑时长等行为数据。数据从PET XML文件经脚本解析转换为CSV格式，最终形成包含完整训练集与三个测试子集的标准化数据集。训练集包含1170条样本，每条记录均携带源文本、机器翻译文本、目标文本以及多达25项行为与质量评估指标。

特点

数据集的核心特色在于其多维度的细粒度行为数据，涵盖击键分类（字母、数字、符号、导航、删除等）、停顿时长（300毫秒与1000毫秒阈值）及编辑时间，同时融合了翻译质量自动评估指标（BLEU、chrF、TER）。特别设计的三个测试子集分别屏蔽了主体身份、翻译模式或时间相关字段，有效避免了评估时的信息泄露，为译员风格计量、模式预测及时间预测等下游任务提供了严谨的基准。

使用方法

数据集通过HuggingFace Datasets库加载，需使用'full'配置并指定本地解压后的数据文件夹路径。支持三种测试配置：test_mask_subject用于译员身份识别任务，test_mask_modality用于翻译模式预测，test_mask_time用于编辑时间与停顿预测。训练集与各测试集均包含item_id、src_text、tgt_text等核心字段，用户可依据任务需求选择相应配置，利用击键与停顿特征进行多标签分类或回归建模，探索译员行为与翻译质量之间的深层关联。

背景与挑战

背景概述

在机器翻译领域，译后编辑（Post-Editing）已成为提升翻译质量的关键环节，然而翻译者的行为特征与编辑风格如何影响最终译文，仍是自然语言处理研究中尚未充分探索的课题。GroNLP/ik-nlp-22_pestyle数据集由荷兰格罗宁根大学信息科学硕士项目的Arianna Bisazza与Gabriele Sarti于2022年创建，旨在通过细粒度的行为数据（如击键、停顿、编辑时长）揭示译后编辑过程中的风格计量学特征。该数据集基于FLORES-101语料库，涵盖英语至意大利语的翻译任务，包含从零翻译与对Google Translate及mBART两种机器翻译系统输出的译后编辑数据。其核心研究问题聚焦于如何通过行为模式识别不同翻译者的编辑风格，为个性化翻译系统与翻译质量评估提供新的视角，对理解人类与机器协作的翻译行为具有重要推动价值。

当前挑战

该数据集面临的核心挑战在于多维度行为数据的复杂表征与隐私保护之间的平衡。首先，在领域问题层面，需解决如何从击键动力学、停顿模式与编辑操作（插入、删除、替换）中提取区分不同翻译者风格的有效特征，避免因标注者数量有限（仅三位）导致的过拟合风险。其次，在构建过程中，数据采集依赖PET平台的精确日志记录，但行为数据的噪声（如非刻意停顿、误操作）与时间戳的粒度差异增加了预处理难度。此外，测试集需通过掩码不同字段（如主体、模态、时间）来防止信息泄露，这要求设计出既能评估模型泛化能力又不破坏数据完整性的划分策略。最后，当前未公开数据的限制进一步增加了研究复现与跨机构合作的障碍，使得风格计量学模型的鲁棒性验证面临挑战。

常用场景

经典使用场景

GroNLP/ik-nlp-22_pestyle数据集的核心应用场景在于细粒度地剖析人工译者在机器翻译后编辑过程中的行为模式。研究者能够利用该数据集中的击键记录、停顿时长、编辑操作序列等行为特征，结合源语言与目标语言的文本对，深入探究译者在不同翻译条件（如从零翻译或对Google Translate、mBART等系统输出进行译后编辑）下的认知负荷与策略差异。该数据集为计算语言学和翻译研究领域提供了一种新颖的量化手段，用以揭示人类语言处理过程中的微观行为与宏观翻译质量之间的内在关联。

解决学术问题

该数据集精准回应了机器翻译后编辑研究中长期存在的两大核心学术问题：一是如何通过行为数据客观度量译者的认知努力，二是如何识别不同译者的个体风格特征。传统研究多依赖主观问卷或粗略的耗时指标，而该数据集通过引入精细的击键分类、停顿分段及编辑操作类型（插入、删除、替换、移位）等多维特征，为量化翻译认知模型提供了可靠的数据基础。此外，它推动了译者风格计量学（post-editing stylometry）这一交叉研究方向的发展，使得研究者能够基于行为数据而非仅依赖最终译文来区分译者身份，从而深化对翻译过程中人类决策机制的理解。

衍生相关工作

该数据集的发布催生了一系列围绕翻译行为预测与译者建模的经典后续工作。基于其提供的多模态行为特征，研究者构建了能够从击键序列与编辑时间中预测译者身份的分类模型，开启了基于行为数据的译者风格识别新范式。此外，相关工作利用该数据集中的停顿与修改操作信息，发展了针对译后编辑努力程度的自动评估方法，将传统仅依赖最终译文质量的评估体系拓展至过程导向的量化分析。这些衍生研究不仅丰富了计算翻译学的理论框架，也为开发更加智能、个性化的计算机辅助翻译系统奠定了实证基础。