meettilavat/Gutenberg_1899

Gutenberg公共领域文本语料库是一个包含57,117个公共领域作品的干净快照，来源于Project Gutenberg。这些作品主要是英文，但也包含一些西欧语言的作品。每个文档都是单个标题的正文（作者逝世于1899年或之前），存储为UTF-8字符串，分布在33个Zstd压缩的Parquet碎片中。去除了模板页眉/页脚，轻量级归一化空白字符，并移除了长度小于1,000字符或ASCII比例低于50%的文本。Gutenberg Public Domain Text Corpus is a cleaned snapshot of 57,117 public-domain works from Project Gutenberg. The texts are primarily in English, but also include works in other Western European languages. Each document is the body text of a single title (authors deceased by 1899), stored as UTF-8 strings in 33 Zstd-compressed Parquet shards. Boilerplate headers/footers are stripped, whitespace is lightly normalized, and texts shorter than 1,000 characters or with an ASCII ratio below 50% are removed.