Etruscan Machine Learning Corpus

The OpenEtruscan ML-Ready Corpus is a normalized, quality-tagged dataset of 6,567 Etruscan inscriptions designed for machine-learning tasks: word and character embedding training, lacuna restoration, glyph recognition, and diachronic analysis.

Each row provides up to 10 columns:

 id - inscription identifier (CIE / ETP / Pallottino-Rix conventions)

raw_text - the carved glyph stream (Old Italic where available)

canonical_transliterated - scholarly transliteration (Bonfante 2002, Wallace 2008 conventions). Greek-block sibilants (θ χ σ φ ξ ς) preserved. Cyrillic / Latin-Extended-B / Math mirror-glyph corruption introduced by upstream OCR has been deterministically mapped to the intended Latin or Greek letter.

canonical_italic -regenerated Old Italic glyph stream (U+10300–U+1032F) for rows where philologically defensible. NULL for Latin-orthography rows (Roman names like PVLFENNIA), retrograde-OCR garbage, and rows containing letters with no Old Italic correspondent (g, y).

canonical_words_only - only intact tokens (no editorial brackets, uncertainty markers, or lacuna dashes). Suitable for word-embedding training where the model should learn from attested whole forms.

translation - English gloss.

year_from / year_to - date range in BCE.

intact_token_ratio - fraction of canonical tokens that are complete (0–1). Filtering knob for ML pipelines.

data_quality - three-class tag: clean / needs_review / ocr_failed.

Quality breakdown:

clean         6,094  (92.8%) - ML-ready

needs_review    154  ( 2.3%) - residual non-standard character

ocr_failed      319  ( 4.9%) - digit-substitution OCR junk, kept for diagnostic / error analysis only

Recommended ML tiers:

Tier 1 (gold) 3,528 rows  - clean ∧ intact_token_ratio = 1.0 ∧ canonical_italic non-NULL. For Old Italic glyph models and highest-quality embeddings.

Tier 2 (clean & intact) 4,058 rows - for word-embedding training.

Tier 3 (any clean)  6,094 rows   - for sequence / lacuna-restoration training where partial words and editorial markup are useful.

Source:

~71% from the Larth dataset (Vico & Spanakis 2023; ID overlap = 4,712);

~29% from the Corpus Inscriptionum Etruscarum (CIE) Vol. I extractions (1,855 rows).

Editorial markup follows Leiden conventions: [ ] for restoration, < > for editorial addition, { } for editorial deletion, ( ) for expansion of abbreviation, ? for uncertainty.

Generated by the openetruscan normalization pipeline:

scripts/data_pipeline/normalize_inscriptions.py

scripts/data_pipeline/merge_larth_metadata.py