遇见数据集

bombman/thwiki-2026-super-clean-3k

收藏
Hugging Face2026-04-26 更新2026-05-03 收录
官方服务:

资源简介:

该数据集是从泰语维基百科(2026年版本)中提取的,专注于高清洁度(Pro-Clean),专为LLM蒸馏和微调设计。关键特征包括:高信息密度(选择长度超过1000字符且泰语比例大于50%的文章)、专业清洁(移除图像文件名、维基符号垃圾如==, 和空括号)、实体保留(保留引号以维护专有名词和地点边界)以及随机化(从整个文件中随机抽样以确保内容多样性)。

This dataset is extracted from Thai Wikipedia (Dump 2026), focusing on high cleanliness (Pro-Clean) and specifically designed for LLM distillation and fine-tuning. Key features include: High Information Density (selecting articles longer than 1,000 characters with Thai language proportion > 50%), Pro-Cleaned (removing image file names, Wiki symbol garbage such as ==, , and empty brackets), Entity Preserved (retaining quotation marks to maintain boundaries of proper nouns and places), and Randomized (randomly sampling from the entire file to ensure content diversity).

提供机构:
bombman
二维码
社区交流群
二维码
科研交流群
商业服务