DatarrX/mywikisource

DatarrX MyWikisource语料库是一个高质量、大规模的缅甸语数据集，提取自缅甸语维基文库，专为自然语言处理（NLP）任务设计，包括语言建模、拼写检查和正式文本分析。数据集经过严格的清洗和处理，包括命名空间过滤、HTML/XML标签清理、语言阈值过滤、去重和噪声消除。数据集结构包含ID、类别、标题和上下文列。数据集采用MIT许可证发布，旨在支持缅甸语NLP生态系统的发展。

The DatarrX MyWikisource Corpus is a high-quality, large-scale Burmese language dataset extracted and refined from the official Burmese Wikisource, specifically designed for Natural Language Processing (NLP) tasks such as language modeling, spell checking, and formal text analysis. The dataset has undergone rigorous cleaning processes including namespace filtering, HTML/XML tag removal, language thresholding, deduplication, and noise mitigation. It features columns for ID, categories, title, and context. Released under the MIT License, this dataset aims to support the Burmese NLP ecosystem.