UWV/Leesplank_NL_wikipedia_simplifications_preprocessed

Leesplank_nl_wikipedia_simplifications_preprocessed数据集是基于荷兰维基百科条目的简化版本，旨在用于微调大型语言模型。数据集经过了预处理，包括移除少于7个单词的提示行、添加指令列、使用Levenshtein距离排序以及随机分割数据集为训练集、验证集和测试集。数据集包含680,142,754个令牌，原始数据集经过去重和分割后，保留了2.69M行数据。数据集的语言为荷兰语，遵循Apache 2.0许可证。

Leesplank_nl_wikipedia_simplifications_preprocessed is a synthetic dataset made by simplifying Dutch Wikipedia entries, aimed at fine-tuning large language models. The dataset has been preprocessed by removing rows with fewer than 7 words on the prompt side, adding an instructions column, sorting the set using Levenshtein distance, and randomly splitting the dataset into training, validation, and test sets. The dataset contains 680,142,754 tokens, with 2.69M rows remaining from the original deduplicated set. The language of the dataset is Dutch, and it is licensed under Apache 2.0.