Source Library Recitation-Refusal Census: which historical printed texts Gemini declines to transcribe (snapshot 2026-10-24)
收藏资源简介:
Source Library (sourcelibrary.org) transcribes printed historical books with Google's Gemini 3.1 Flash-Lite. Gemini sometimes returns nothing with the finish reason RECITATION, judging that the transcription would reproduce text it treats as already published. This census counts those refusals book by book: 56,854 refused pages out of 2,240,424 attempted, on 17,519 books in English, Latin, German, French and Italian. Files. books.jsonl: 8,955 public books with year, language, edition and work keys, Internet Archive identifier and download count, Project Gutenberg matches, whether the Archive's own clean OCR text of the edition is online, and pages attempted, refused, and refused-then-read. summary.csv: counts by language, period and Archive-text status over all 17,519 books. DATASHEET.md: definitions, method, limits. Findings. English print is refused on 4-15% of pages in every period; Latin, German and French before 1800 on 0.03-0.4%. Books whose clean Archive text is online are refused on 13.7% of attempted pages against 1.8% for books without it. An image-free continuation test (55 sealed pages) found no sign that refused pages are better memorised by the models, so a refusal is evidence that the text is held online, not of training-set membership. Limits. One engine; the pipeline chose which pages to send (pages the Archive text could fill were mostly not sent); refusals are the pipeline's stamps, not re-runs. See DATASHEET.md. Built by scripts/eval/estimator/ocr/refusal-census.mjs and scripts/eval/export-refusal-census.mjs in the sourcelibrary-v2 repository (AGPL-3.0); discussion in issues #6740 and #7271. Data licence: CC BY 4.0. No reader or personal data.



