Kiy-K/pretraining-corpus

Kiy-K合成预训练语料库是一个大规模的合成英文文本集合，旨在用于语言模型的预训练和指令调整研究。所有数据均为合成数据，由开源大型语言模型生成，不包含任何真实用户、版权或敏感信息。语料库覆盖了多个领域，包括科技编程、科学教育、日常对话以及问答式文本。

The Kiy-K Synthetic Pretraining Corpus is a large-scale collection of synthetically generated English text designed for language model pretraining and instruction-tuning research. All data is synthetic, created using open-source large language models and does not contain any real user, copyrighted, or sensitive information. The corpus covers diverse domains including technology and programming, science and education, general conversation, and QA-style texts.