OCR fulltexts of the Digital Collections of the Berlin State Library (DC-SBB)

The digital collections of the SBB contain 153,942 digitized works from the time period of 1470 to 1945. At the time of publication, 28,909 works have been OCR-processed resulting in 4,988,099 full-text pages. For each page with OCR text, the language has been determined by langid (Lui/Baldwin 2012). corpus-entropy.pkl entropy rate per document page corpus-language.pkl language per document page corpus.zip fulltext corpus (extracts to .txt format) de_corpus.zip German sub-corpus (extracts to .txt format) selection_de.pkl Selection list of German documents xml2csv_alto.csv fulltext corpus per document page (incl.OCR word confidences) Sources Marco Lui and Timothy Baldwin. 2012. Langid.py: An off-the-shelf language identification tool. In Proceedings of the ACL 2012 System Demonstrations, ACL ’12, pages 25–30, Stroudsburg, PA, USA. Association for Computational Linguistics