Bretagne/fineweb-2_raw_breton
收藏官方服务：
资源简介：
fineweb-2数据集是于2024年12月推出的，包含两种分割方式的布列塔尼语数据：bre_Latn_removed和bre_Latn。数据集由CommonCrawl的文件中提取的文本构成，经过语言分类、特定语言的过滤器筛选和MinHash去重处理。作者提供了数据集的原始版本，以便用户可以根据自己的需求进行过滤和去重。
The fineweb-2 dataset, introduced in December 2024, contains two splits in Breton: bre_Latn_removed and bre_Latn. The dataset is composed of texts extracted from CommonCrawl files, which have been processed through language classification, language-specific filtering, and MinHash deduplication. The authors have provided the raw version of the dataset for users to apply their own filtering and deduplication methods.
提供机构：
Bretagne
