EvaCun: ORACC Akkadian Parallel Corpus

EvaCun: ORACC Akkadian Parallel Corpus for Machine Translation (train/validation), v0.1

DESCRIPTION

Overview:This dataset provides aligned Akkadian (transliteration), Akkadian (Unicode cuneiform), and English segments prepared for machine translation experiments in the EvaCun workflow. Data are sourced from ORACC project exports and organized for reproducible training, tokenization, and evaluation with the EvaCun/Akkademia pipeline.

Contents:Six plain-text files (UTF-8), one segment per line, aligned by row index across languages within each split.

akkadian_train.txt

transcription_train.txt

english_train.txt

akkadian_validation.txt

transcription_validation.txt

english_validation.txt

Note on terms:“Transcription” denotes standardized Romanized Akkadian used by downstream tools. “Akkadian (Unicode cuneiform)” follows the EvaCun conversion stage described in the workflow.

Source and preprocessing:The corpus originates from ORACC project data (“corpusjson” exports). Preprocessing in the EvaCun pipeline includes:

cleaning of spurious markers (e.g., replacing stray NaN / Uncertain);

normalization of gap tokens (X / ...);

per-text segmentation; and

alignment of transliteration, cuneiform, and English lines under a shared id_text.The split strategy follows an 80/10/10 design (train/validation/test). The test portion is withheld for separate evaluation. This deposit includes train and validation only.

Intended use:Neural MT experiments and reproducible benchmarking with the EvaCun/Akkademia stack. The accompanying notebooks demonstrate tokenization with SentencePiece and model training/translation with fairseq (train.py, generate.py), in line with the documented workflow.

Format:

Encoding: UTF-8 (Unix line endings)

Alignment: strict line-wise alignment within each split (row n in each file corresponds to the same segment)

Headers: none; plain text only

Limitations and notes:Coverage reflects the subset of ORACC Akkadian projects processed to date; genre distribution may be uneven. Alignments are sentence/line-level per the workflow heuristics and may require task-specific re-segmentation. Users should review domain/genre needs before training.

Versioning:v0.1 corresponds to the dataset prepared for the EvaCun Colab Notebook release described in the workflow notes. Future versions may expand coverage or revise segmentation.

Related resources:EvaCun Colab notebooks and setup guides (sister repositories) that consume this dataset and reproduce the pipeline stages (tokenization, training, translation, detokenization):https://github.com/ancient-world-citation-analysis/EvaCun-Colab-Notebook/tree/main

Keywords:Akkadian; cuneiform; ORACC; machine translation; parallel corpus; SentencePiece; fairseq; EvaCun

License:CC BY 1.0 (N.B redistribution is permitted for the specific ORACC derivatives we prepared).

SHORT DATASET CARD

EvaCun: ORACC Akkadian Parallel Corpus (v0.1). A line-aligned, three-column parallel corpus (Akkadian transliteration / Akkadian Unicode cuneiform / English) prepared from ORACC sources for MT. Preprocessing and the training workflow (tokenization with SentencePiece; training/translation with fairseq) follow the EvaCun/Akkademia notes. Provided files: six UTF-8 .txt for train/validation splits; one segment per line; aligned by index. Intended for MT training and reproducible benchmarking with the EvaCun Colab notebooks. Limitations: coverage and genre balance reflect available ORACC data; alignment and normalization decisions may affect downstream scores. Cite this dataset’s Version DOI; see the GitHub repository above for notebooks and pipeline details.

HOW TO CITE (DATASET, VERSION-SPECIFIC — RECOMMENDED)

APA (human-readable):Anderson, A. (2025). EvaCun: ORACC Akkadian Parallel Corpus for Machine Translation (train/validation), v0.1 [Dataset]. Zenodo. https://doi.org/10.5281/zenodo.17220688

BibTeX:@dataset{evacun_oracc_parallel_v01_2025,title = {EvaCun: ORACC Akkadian Parallel Corpus for Machine Translation (train/validation), v0.1},author = {Anderson, Adam},year = {2025},publisher = {Zenodo},version = {v0.1},doi = {10.5281/zenodo.17220688},url = {https://doi.org/10.5281/zenodo.17220688},note = {Contributors: Christian Karren (Project manager), Olivia McCauley (Project manager), Melanie Her (Data collector), Mackenzie Moffit (Project member), Lirui Harrison Huang (Data curator), Jenna Kudaimi (Data collector), Neha Ramakrishnan (Data collector), Aurelia Widjaja (Project member), Mohammad Shahnawaz (Project member), Emily Xu (Data collector), Indu Abhilash (Data collector).}}

CREATORS

Anderson, Adam (University of California, Berkeley)

CONTRIBUTORS

Karren, Christian (University of California, Berkeley) — Project manager

McCauley, Olivia (University of California, Berkeley) — Project manager

Her, Melanie (University of California, Berkeley) — Data collector

Moffit, Mackenzie (University of California, Berkeley) — Project member

Huang, Lirui Harrison (University of California, Berkeley) — Data curator

Kudaimi, Jenna (University of California, Berkeley) — Data collector

Ramakrishnan, Neha (University of California, Berkeley) — Data collector

Widjaja, Aurelia (University of California, Berkeley) — Project member

Shahnawaz, Mohammad (University of California, Berkeley) — Project member

Xu, Emily (University of California, Berkeley) — Data collector

Abhilash, Indu (University of California, Berkeley) — Data collector

EvaCun: ORACC阿卡德语机器翻译平行语料库（训练/验证集）v0.1

### 数据集描述

#### 概述
本数据集为适配EvaCun工作流中的机器翻译实验构建，提供对齐的阿卡德语（转写）、阿卡德语（Unicode楔形文字）及英语语片段。数据源自ORACC项目导出文件，经标准化组织后，可用于EvaCun/Akkademia工作流的可复现训练、分词与评估。

#### 数据内容
本数据集包含6个UTF-8编码的纯文本文件，每行对应一个语片段；每个划分集内的跨语言文件按行索引严格对齐。具体文件如下：
akkadian_train.txt、transcription_train.txt、english_train.txt、akkadian_validation.txt、transcription_validation.txt、english_validation.txt

#### 术语说明
“转写（Transcription）”指下游工具所使用的标准化罗马化阿卡德语。“阿卡德语（Unicode楔形文字）”遵循工作流中描述的EvaCun转换阶段规范。

#### 来源与预处理
本语料库源自ORACC项目的“corpusjson”导出数据。EvaCun工作流中的预处理步骤包括：
1. 清理虚假标记（例如替换游离的NaN/不确定标记）；
2. 归一化空缺标记（X/...）；
3. 按文本进行分段；
4. 基于共享id_text对齐转写、楔形文字与英语行。

本数据集的划分策略遵循80/10/10比例（训练集/验证集/测试集），测试集留作独立评估使用，本次发布仅包含训练集与验证集。

#### 预期用途
适用于基于EvaCun/Akkademia工具栈的神经机器翻译实验与可复现基准测试。配套的Notebook演示了使用SentencePiece进行分词，以及使用fairseq（train.py、generate.py）完成模型训练与翻译，与已记录的工作流完全一致。

#### 格式规范
- 编码：UTF-8（采用Unix换行符）
- 对齐方式：每个划分集内严格按行对齐（每个文件的第n行对应同一语片段）
- 无表头，仅包含纯文本内容

#### 局限性与说明
语料覆盖范围仅反映当前已完成处理的ORACC阿卡德语项目子集，体裁分布可能存在不均衡情况。本次对齐基于工作流的启发式规则，为句子/行级对齐，可能需要针对特定任务进行重新分段。用户在开展训练前，应先评估自身的领域/体裁需求。

#### 版本说明
v0.1版本对应工作流说明中发布的EvaCun Colab Notebook所用数据集，未来版本可能扩展覆盖范围或修订分段规则。

#### 相关资源
EvaCun Colab Notebooks及设置指南（姊妹仓库），可使用本数据集复现工作流各阶段（分词、训练、翻译、去分词）：https://github.com/ancient-world-citation-analysis/EvaCun-Colab-Notebook/tree/main

#### 关键词
阿卡德语；楔形文字（cuneiform）；ORACC；机器翻译；平行语料库；SentencePiece；fairseq；EvaCun

#### 授权协议
CC BY 1.0（注：本数据集所包含的ORACC衍生内容允许进行再分发）

### 数据集简要说明卡片
EvaCun: ORACC阿卡德语平行语料库（v0.1）。该语料库为行对齐的三列平行语料库（阿卡德语转写 / 阿卡德语Unicode楔形文字 / 英语），源自ORACC数据源，专为机器翻译任务构建。预处理与训练工作流（使用SentencePiece分词；使用fairseq开展训练/翻译）遵循EvaCun/Akkademia官方说明。本次发布提供6个UTF-8编码的.txt文件，对应训练/验证划分集；每行一个语片段，按索引完成对齐。本数据集旨在用于EvaCun Colab Notebooks的机器翻译训练与可复现基准测试。局限性：语料覆盖范围与体裁平衡反映了当前可用的ORACC数据情况；对齐与归一化决策可能对下游评测分数产生影响。引用本数据集时请使用其版本DOI；详见上述GitHub仓库获取Notebook与工作流细节。

### 引用方式（数据集版本专属，推荐）
#### APA格式（人类可读版）：
Anderson, A. (2025). EvaCun: ORACC阿卡德语机器翻译平行语料库（训练/验证集）v0.1 [数据集]. Zenodo. https://doi.org/10.5281/zenodo.17220688

#### BibTeX格式：
@dataset{evacun_oracc_parallel_v01_2025,
title = {EvaCun: ORACC阿卡德语机器翻译平行语料库（训练/验证集）v0.1},
author = {Anderson, Adam},
year = {2025},
publisher = {Zenodo},
version = {v0.1},
doi = {10.5281/zenodo.17220688},
url = {https://doi.org/10.5281/zenodo.17220688},
note = {贡献者：Christian Karren（项目负责人）、Olivia McCauley（项目负责人）、Melanie Her（数据采集员）、Mackenzie Moffit（项目成员）、Lirui Harrison Huang（数据管理员）、Jenna Kudaimi（数据采集员）、Neha Ramakrishnan（数据采集员）、Aurelia Widjaja（项目成员）、Mohammad Shahnawaz（项目成员）、Emily Xu（数据采集员）、Indu Abhilash（数据采集员）。}
}

### 创作者
Anderson, Adam（加州大学伯克利分校）

### 贡献者
- Karren, Christian（加州大学伯克利分校）——项目负责人
- McCauley, Olivia（加州大学伯克利分校）——项目负责人
- Her, Melanie（加州大学伯克利分校）——数据采集员
- Moffit, Mackenzie（加州大学伯克利分校）——项目成员
- Huang, Lirui Harrison（加州大学伯克利分校）——数据管理员
- Kudaimi, Jenna（加州大学伯克利分校）——数据采集员
- Ramakrishnan, Neha（加州大学伯克利分校）——数据采集员
- Widjaja, Aurelia（加州大学伯克利分校）——项目成员
- Shahnawaz, Mohammad（加州大学伯克利分校）——项目成员
- Xu, Emily（加州大学伯克利分校）——数据采集员
- Abhilash, Indu（加州大学伯克利分校）——数据采集员