BramVanroy/CommonCrawl-CreativeCommons-fine

Common Crawl Creative Commons Corpus Fine (C5f) 是 Common Crawl Creative Commons Corpus (C5) 的一个过滤版本，只保留那些也出现在 FineWeb 或 FineWeb-2 数据集中的样本。该数据集用于文本生成任务和语言建模。README 列出了各种配置的特定数据文件以及数据集中包含的语言。它还提供了数据集大小、文档数量和标记数量的详细信息。该数据集在知识共享许可下。

Common Crawl Creative Commons Corpus Fine (C5f) is a filtered version of the Common Crawl Creative Commons Corpus (C5), retaining only those samples present in the FineWeb or FineWeb-2 datasets. It is designed for text generation tasks and language modeling. The README lists various configurations with specific data files and the languages included in the dataset. It also provides details about the dataset size, the number of documents, and the number of tokens. The dataset is licensed under Creative Commons.