caskcsg/LongMagpie_p-mix_64k_dataset

LongMagpie是一个自合成框架，用于自动生成大规模的长上下文指令数据。该数据集通过利用对齐的长上下文大型语言模型（LLMs）来自动回归生成上下文相关的查询，从而无需人工干预即可生成高质量的指令。实验表明，LongMagpie在长上下文任务上取得了领先性能，同时在短上下文任务上也能保持竞争力，使其成为开放、多样化和可扩展的长上下文指令数据合成的有效方法。

LongMagpie is a self-synthesis framework that automatically generates large-scale long-context instruction data. The dataset utilizes aligned long-context large language models (LLMs) to auto-regressively generate contextually relevant queries, producing high-quality instructions without human effort. Experiments demonstrate that LongMagpie achieves leading performance on long-context tasks while maintaining competitive performance on short-context tasks, establishing it as a simple and effective approach for open, diverse, and scalable long-context instruction data synthesis.