five

DatarrX/myX-Burmese-Synthetic-Pseudo-Syllables

收藏
Hugging Face2026-05-24 更新2026-04-12 收录
下载链接:
https://hf-mirror.com/datasets/DatarrX/myX-Burmese-Synthetic-Pseudo-Syllables
下载链接
链接失效反馈
官方服务:
资源简介:
该数据集包含5,814,699个计算机生成的(合成)缅甸语伪音节,这些音节基于特定的复杂语言和正字法模式系统化构建。作为低资源语言处理基础研究的一部分,该数据集为缅甸语自然语言处理(NLP)、分词、字体渲染和拼写检查系统提供了严格的基准和压力测试环境。数据集通过组合管道生成,基于缅甸字母表的结构:定义了超过267个基础结构模式,使用占位符字符အ与各种中间音、元音和变音符号组合;对于每个模式,占位符အ被系统地替换为所有33个核心缅甸语辅音(从က到အ);这种数学替换使数据集能够映射出在脚本规则内所有有效、边缘和结构极端的字素堆叠可能性。需要注意的是,由于数据集是通过程序化生成以覆盖组合可能性,因此包含标准缅甸语词典中不存在的音节、日常交流中未使用的模式以及古老或不存在的语言字符串。不建议盲目使用该数据集训练标准文本生成的语言模型,以免引入虚构、不存在的单词(模型中毒);而应将其用于负采样、边缘情况字体渲染测试、压力测试音节分割器以及训练拼写检查器以识别边界。

This dataset contains 5,814,699 computer-generated (synthetic) Burmese pseudo-syllables, systematically constructed based on specific complex linguistic and orthographic patterns. As part of foundational research on low-resource language processing, this dataset provides rigorous benchmarking and stress-testing environments for Burmese natural language processing (NLP), syllable segmentation, font rendering, and spell-checking systems. The dataset is generated via a combinatorial pipeline rooted in the structure of the Burmese alphabet: over 267 basic structural patterns are defined, which pair the placeholder character အ with various medials, vowels, and diacritics. For each pattern, the placeholder အ is systematically replaced with all 33 core Burmese consonants, ranging from က to အ. This mathematical substitution enables the dataset to map all valid, marginal, and structurally extreme grapheme stacking possibilities compliant with the script's rules. It is important to note that, as the dataset is programmatically generated to fully cover combinatorial possibilities, it includes syllables absent from standard Burmese dictionaries, patterns unused in daily communication, and archaic or non-existent linguistic strings. It is not recommended to blindly use this dataset to train standard text-generating language models, as this risks introducing fictitious, non-existent lexical items (model poisoning). Instead, the dataset should be employed for negative sampling, edge-case font rendering tests, stress-testing syllable segmenters, and training spell-checkers to identify boundary cases.
提供机构:
DatarrX
二维码
社区交流群
二维码
科研交流群
商业服务