KletterMix/KletterMix

KletterMix3是一个德语文本数据集，规模在100M到1B之间，基于上游数据集nvidia/Nemotron-ClimbMix构建。该数据集仅包含文本数据，以分片JSONL格式存储，支持流式访问。数据特征包括cluster_id（聚类ID）、text（文本内容）、token_count（使用GPT-2 tokenizer计算的token数量）和proxy_score（代理分数）。需要注意的是，该数据集目前处于进行中状态，不是最终版本。KletterMix3 is a German text dataset with a size between 100M and 1B, built from the upstream dataset nvidia/Nemotron-ClimbMix. It contains only text data, stored in sharded JSONL format for streaming-friendly access. The dataset features include cluster_id, text, token_count (calculated using GPT-2 tokenizer), and proxy_score. Note that this dataset is currently a Work In Progress and not the final upload.