chizk/wikipedia-pretrain-zh-tw

台灣正體中文維基百科（zh-tw Wikipedia）数据集，每月从维基百科dump获取，内容源自简体中文维基百科（与wikipedia-pretrain-zh项目一致），经OpenCC转换为繁体中文，避免了繁简混杂问题。数据集包含经过筛选的维基百科文本段落，每个条目以JSON格式存储，包含“title”（段落标题，通常为“主题 - 章节”格式）和“text”（文本内容）字段。仅保留知识性段落，去除图片、表格等非知识内容，并经过长度与质量筛选。适用于自然语言处理任务，如预训练模型。授权为Apache-2.0。

Traditional Chinese Wikipedia (zh-tw Wikipedia) dataset, obtained monthly from Wikipedia dumps, with content sourced from Simplified Chinese Wikipedia (consistent with the wikipedia-pretrain-zh project) and converted to Traditional Chinese using OpenCC to avoid mixed simplified and traditional characters. The dataset contains filtered Wikipedia text paragraphs, each stored in JSON format with fields including title (paragraph title, typically in topic - section format) and text (text content). Only meaningful knowledge-based paragraphs are retained, removing non-knowledge content such as images and tables, and subjected to length and quality filtering. Suitable for natural language processing tasks, such as pre-training models. Licensed under Apache-2.0.