tokyotech-llm/swallow-code-v2

SwallowCode-v2 是一个包含 498 亿个标记的 Apache-2.0 许可证授权的 Python 代码数据集，它是由 The-Stack-v2 数据集通过 LLM 重写流程生成的。该数据集旨在用于可扩展的 LLM 预训练，所有样本都已自动格式化、风格规范化，并通过 LLM 重写流程增强了算法清晰度。数据集的构建过程包括自动格式化、长度过滤、LLM 质量评分、LLM 重写阶段和最终格式化。SwallowCode-v2 在下游代码生成基准测试中表现出比其他开源代码数据集更强的性能。

SwallowCode-v2 is a 49.8 billion-token Apache-2.0-licensed Python code dataset rewritten from The-Stack-v2 using an LLM rewriting pipeline. The dataset is designed for scalable LLM pre-training, with all samples auto-formatted, style-normalized, and enhanced for algorithmic clarity. The dataset creation process includes auto-formatting, length filtering, LLM quality scoring, LLM rewriting, and final formatting. SwallowCode-v2 demonstrates stronger performance than other open-source code datasets on downstream code-generation benchmarks.