OCR fulltexts of the Digital Collections of the Berlin State Library (DC-SBB)

The digital collections of the SBB contain 153,942 digitized works from the time period of 1470 to 1945.

At the time of publication, 28,909 works have been OCR-processed resulting in 4,988,099 full-text pages.
For each page with OCR text, the language has been determined by langid (Lui/Baldwin 2012).

corpus-entropy.pkl      entropy rate per document page

corpus-language.pkl   language per document page

corpus.zip                    fulltext corpus (extracts to .txt format)

de_corpus.zip              German sub-corpus (extracts to .txt format)

selection_de.pkl          Selection list of German documents

xml2csv_alto.csv         fulltext corpus per document page (incl.OCR word confidences)

Sources

Marco Lui and Timothy Baldwin. 2012. Langid.py:

An off-the-shelf language identification tool. In Proceedings of the ACL 2012 System Demonstrations,

ACL ’12, pages 25–30, Stroudsburg, PA, USA. Association for Computational Linguistics