ali-issa/arb_aya_dataset

Aya数据集由16个适合训练英语和阿拉伯语两个大型语言模型的子数据集组成。原始数据集包含101种语言的19个子数据集，但经过筛选后只使用了16个。由于翻译不完整和存在错误，CNN Daily Mail、XLEL-WD和SODA数据集被排除。The Aya dataset comprises 16 selected sub-datasets suitable for training two large language models, one for English and one for Arabic. The original dataset included 19 sub-datasets translated into 101 languages, but only 16 were used after selection. The CNN Daily Mail, XLEL-WD, and SODA datasets were excluded due to incomplete translations and translation errors.