Source Library quality dataset v1: OCR and translation quality evidence for historical primary sources
收藏资源简介:
The per-page evidence behind the draft paper How page quality is measured (Source Library, sourcelibrary.org), which asks of a translated historical page: does this English say what is printed on this leaf? With these files and no access to our database, a reader can recompute every figure the paper takes from them. Translation audit (audit-pages.jsonl, audit-controls.jsonl, audit-weights.json): one random interior page from each of 311 books in 15 languages, with the served transcription and English translation, a Claude Opus fidelity rating (1-5) with typed defects, a second judge (Claude Sonnet) on 107 pages, 45 blinded swap, drop and repeat controls, and the weights for the language-weighted estimate. Measure: judged (no reference, no image). Eye check (eye-check.jsonl): 20 audited pages read against the scan by a model (Claude). Measure: accuracy. Two-read screen (two-read-screen.jsonl): 327 pages, served OCR against three fresh reads, with the judge's garble label. Measure: agreement. OCR accuracy (ocr-accuracy-cells.json, ocr-accuracy-latin-pages.jsonl): character error rate against published references for the cells the paper quotes, with the per-page Latin rows. Reference windows (references/): 298 CC BY-SA 4.0 windows (Perseus, First1KGreek, Kanripo, Wikisource) and 32 US public-domain windows that are also public domain in the EU, with source, revision and work for attribution; 209 more as pointers (source URL, revision, sha256) where the licence restricts redistribution, is not recorded, or the work is not public domain in the EU. Withheld text. The transcription and translation of 12 audit pages from editions that are in copyright, or not established as public domain in the EU, are withheld; their scores, verdicts and hashes remain. See LICENSES.md. Limits. The judge never saw the scan; judged is not accuracy; many reference windows are canonical texts that models may recite; most languages have under 30 audited books. See README.md. Licence. CC BY-SA 4.0 for our data; third-party reference text keeps its own licence, named per file and per row. Contains a canary GUID (CANARY.txt): do not train on this data. Files in references/ are named references__file on Zenodo. Built by scripts/eval/build-quality-dataset.mjs in the sourcelibrary-v2 repository (AGPL-3.0), which regenerates it from committed files and checks every quoted figure; issues #5531 and #7462. No reader or personal data.



