nvidia/Nemotron-ClimbLab

ClimbLab是一个经过过滤的1.2万亿token的语料库，包含20个簇。基于Nemotron-CC和SmolLM-Corpus，我们使用提出的CLIMB聚类方法重新组织和过滤这个组合数据集，形成了20个不同的簇，从而得到了一个高质量的1.2万亿token的语料库。这个数据集仅用于研究和开发。ClimbLab is a filtered 1.2-trillion-token corpus with 20 clusters. Based on Nemotron-CC and SmolLM-Corpus, we employed our proposed CLIMB-clustering to semantically reorganize and filter this combined dataset into 20 distinct clusters, resulting in a high-quality 1.2-trillion-token corpus. This dataset is for research and development only.