Korean and English Pretraining Corpus
收藏arXiv2025-09-30 收录
下载链接：
https://huggingface.co/kifai/GECKO-7B
下载链接
链接失效反馈官方服务：
资源简介：
该数据集是一个平衡的韩英文本语料库，旨在为双语大型语言模型优化模型训练。该语料库包含了来自多个来源的数据，特别注重减轻有害内容并保持结构完整性，以提高模型性能。其规模达到数太字节，任务是对双语语言模型进行预训练。
This dataset is a balanced Korean-English text corpus developed to optimize model training for bilingual large language models. It collects data from multiple sources, with special emphasis on mitigating harmful content and maintaining structural integrity to enhance model performance. With a scale of several terabytes, this corpus is intended for pre-training bilingual language models.
提供机构：
Open-source community
