Honkoku-Lines: A Million-Scale Line-Level Image–Transcription Dataset for Historical Japanese OCR Built from Crowdsourced Transcriptions

This record archives the construction source code and the line-level companion metadata for Honkoku-Lines, a line-level dataset for handwritten kuzushiji (cursive historical Japanese) text recognition (OCR/HTR), built from crowdsourced transcriptions (honkoku) aligned to IIIF page images.

The line images and the full dataset are published on Hugging Face: https://huggingface.co/datasets/yuta1984/honkoku-lines. This Zenodo record provides the accompanying artifacts for the data paper:

src.zip — the complete as-run construction pipeline (IIIF page retrieval, layout detection, dual line cropping, PARSeq OCR, edit-distance alignment with gold transcriptions, WebDataset packing, image normalisation, optional enrichment, and layout-consistency cleaning), organised in execution order.

lines.jsonl.gz — per-line metadata for the clean release (1,163,307 lines; no images): item/image/line identifiers, bounding box, IIIF region URL, koji-notation transcription, plain text, reference OCR output, edit distance, image license, holding institution, and train/val/test split.

items.tsv — per-item bibliography, statistics, and license (4,197 items).

hosts.tsv — per-institution aggregates (32 holding institutions).

koji_notation.md — reference for the diplomatic (koji) markup used in the transcription field.

README.md — documentation of the files, schemas, line-source and license breakdowns, and usage examples for this record.

License (multi-licensed collection). The construction source code (src.zip) is released under the MIT License; the transcription text and line-level metadata (lines.jsonl.gz, items.tsv, hosts.tsv, koji_notation.md, README.md) under CC BY-SA 4.0. Line images (on Hugging Face) follow each holding institution's terms, recorded per item in items.tsv; the CC-BY-4.0 and PDM-1.0 subset accounts for the large majority of lines (86.6%).