gair-prox/c4-pro
收藏Hugging Face2024-09-26 更新2024-12-14 收录
下载链接：
https://hf-mirror.com/datasets/gair-prox/c4-pro
下载链接
链接失效反馈官方服务：
资源简介：
c4-pro数据集是通过ProX框架从c4数据集精炼而来的，包含了大约400亿个高质量的词元，适用于通用语言模型的预训练。
The c4-pro dataset is refined from the c4 dataset using the ProX refining framework, containing about 40B high-quality tokens, suitable for general language model pre-training. The dataset is based on the ODC-By 1.0 license and requires users to abide by the CommonCrawl terms of use.
提供机构：
gair-prox
