smgkz/swallow-code

SwallowCode 是一个高质量的代码数据集，包含约161亿个Python代码令牌，源自The-Stack-v2-train-smol-ids数据集，通过四阶段管道处理：语法验证、基于pylint的样式过滤，以及使用Llama-3.3-70B-Instruct的两阶段大型语言模型（LLM）重写过程。该管道强制执行样式一致性（Style-Guided Code Rewriting, SGCR），并将代码片段转换为自包含、算法高效的示例（Self-Contained Optimization Rewriting, SCOR）。SwallowCode旨在提升大型语言模型在程序合成和代码生成方面的性能。数据集经过精心管理，以消除噪声、确保风格一致性并增强语义质量，适用于代码生成任务的预训练和微调。

SwallowCode is a high-quality code dataset comprising approximately 16.1 billion tokens of Python code, derived from The-Stack-v2-train-smol-ids through a four-stage pipeline: syntax validation, pylint-based style filtering, and a two-stage LLM rewriting process using Llama-3.3-70B-Instruct. The pipeline enforces style conformity (Style-Guided Code Rewriting, SGCR) and transforms snippets into self-contained, algorithmically efficient examples (Self-Contained Optimization Rewriting, SCOR). SwallowCode is designed to enhance large language model (LLM) performance in program synthesis and code generation.