starcoderdata
收藏huggingface.co2025-03-23 收录
下载链接：
https://huggingface.co/datasets/bigcode/starcoderdata
下载链接
链接失效反馈官方服务：
资源简介：
StarCoder Training Dataset
Dataset description
This is the dataset used for training StarCoder and StarCoderBase. It contains 783GB of code in 86 programming languages, and includes 54GB GitHub Issues + 13GB Jupyter notebooks in scripts and text-code pairs,
and 32GB of GitHub commits, which is approximately 250 Billion tokens.
Dataset creation
The creation and filtering of The Stack is explained in the original dataset, we additionally decontaminate… See the full description on the dataset page: https://huggingface.co/datasets/bigcode/starcoderdata.
StarCoder 训练数据集
数据集描述
本数据集用于StarCoder及其基础模型的训练，其中包含来自86种编程语言的783GB代码，并涵盖54GB的GitHub Issues及13GB的Jupyter笔记本，以脚本和文本代码对的形式呈现，此外还包括32GB的GitHub提交记录，总计约2500亿个tokens。
数据集创建
关于Stack的创建与筛选过程已在原始数据集的描述中详细说明，此外我们还进行了额外的净化工作...请访问数据集页面以获取完整描述：https://huggingface.co/datasets/bigcode/starcoderdata.
提供机构：
huggingface.co
