jrast/deduped-text-v1
收藏Hugging Face2026-05-26 更新2026-05-31 收录
下载链接：
https://hf-mirror.com/datasets/jrast/deduped-text-v1
下载链接
链接失效反馈官方服务：
资源简介：
这是一个名为Clean Text Corpus的文本数据集，来源于s3://jrast-10605-f26/deduped_text_v1/。数据集包含4,199,571个文档，总计30,441,389,227个字符。它由清理后的HTML parquet文件重新打包成zstd压缩的parquet分片格式。
This is a text dataset named Clean Text Corpus, sourced from s3://jrast-10605-f26/deduped_text_v1/. The dataset contains 4,199,571 documents with a total of 30,441,389,227 characters. It was repackaged from cleaned HTML parquet into zstd-compressed parquet shards.
提供机构：
jrast
