agentlans/HuggingFaceFW-finewiki-sample

这是一个从HuggingFaceFW/finewiki数据集中随机抽取的子集，包含超过一百万页的多种语言Wikipedia文章。数据集的目的是为了提供一个小型、易于管理的分析、微调和基准测试数据集。语言的选择基于页面数量和内容质量。对于每种选定的语言，随机选择了1%的文章并进行随机排序。数据集包括一个聚合了所有语言样本的assorted配置。

This is a subset randomly extracted from the HuggingFaceFW/finewiki dataset, containing Wikipedia articles in multiple languages with over one million pages. The dataset aims to provide a smaller and more manageable set for analysis, fine-tuning, and benchmarking. The selection of languages is based on page count and content quality. For each selected language, 1% of the articles are randomly chosen and shuffled. The dataset includes an assorted configuration that aggregates samples from all languages.