bombman/thwiki-2026-super-clean-10k
收藏Hugging Face2026-04-26 更新2026-05-03 收录
下载链接:
https://hf-mirror.com/datasets/bombman/thwiki-2026-super-clean-10k
下载链接
链接失效反馈官方服务:
资源简介:
该数据集是从泰国维基百科(2026年版)中提取的,专注于“专业清洁”级别,专门用于蒸馏和微调大型语言模型。关键特性包括:高信息密度(只选择文章长度超过1000个字符且泰语比例大于50%)、专业清洁(移除图片文件名、维基符号垃圾和空括号)、实体保留(保留引号以保持专有名词和地点的边界)以及随机化(从整个文件中随机抽取数据以确保内容多样性)。
This dataset is extracted from the Thai Wikipedia (Dump 2026), focusing on Pro-Clean level cleanliness, specifically designed for distillation and fine-tuning LLMs. Key features include: High Information Density (selecting only articles with over 1,000 characters and Thai language proportion > 50%), Pro-Cleaned (removing image file names like .jpg, .png, Wiki symbol garbage such as ==, "", and empty brackets), Entity Preserved (retaining quotation marks to preserve boundaries of proper nouns and places), and Randomized (randomly sampling data from the entire file to ensure content diversity).
提供机构:
bombman



