OpenCoder-LLM/RefineCode-code-corpus-meta

该数据集包含来自RefineCode的原始代码数据的元信息，如仓库名称、文件路径等。这些元信息可以用于收集和重现RefineCode。目前，数据集只上传了The Stack V2覆盖的元数据，占文件总量的约50%。RefineCode是一个高质量、可重现的代码预训练语料库，包含9600亿个标记，覆盖607种编程语言，并包含从网络语料库中提取的750亿个代码相关标记。

This dataset consists of meta information (including the repository name and file path) of the raw code data from RefineCode. You can collect those files referring to this metadata and reproduce RefineCode! Currently, we have uploaded the meta data covered by The Stack V2 (About 50% file volume). RefineCode is a high-quality, reproducible code pretraining corpus comprising 960 billion tokens across 607 programming languages and 75 billion code-related token recalled from web corpus.