dclm_100BT
收藏魔搭社区2026-05-13 更新2026-05-17 收录
下载链接：
https://modelscope.cn/datasets/HuggingFaceFW/dclm_100BT
下载链接
链接失效反馈官方服务：
资源简介：
# DCLM 100BT
A ~100 billion token English subset of [DCLM-Baseline 1.0](https://huggingface.co/datasets/mlfoundations/dclm-baseline-1.0-parquet), created for efficient pretraining experiments.
Part of the [Smol-Data](https://huggingface.co/collections/HuggingFaceFW/smol-data) collection — tried and tested mixes for strong pretraining.
## Dataset Description
This dataset was created by randomly sampling from the full DCLM-Baseline 1.0 dataset (~3.5T tokens) to produce a ~100B token subset. Sampling was performed with a fixed seed (42) and a slight 1.05× oversampling factor to account for variance.
A pre-shuffled version is available at [HuggingFaceFW/dclm_100BT-shuffled](https://huggingface.co/datasets/HuggingFaceFW/dclm_100BT-shuffled).
## How It Was Created
The dataset was generated using [datatrove](https://github.com/huggingface/datatrove) with the [smol_data.py](https://github.com/huggingface/datatrove/blob/main/examples/smol_data.py) script. The pipeline reads from the source dataset in streaming mode, applies a `SamplerFilter` to downsample, and writes the result back to the Hugging Face Hub.
## Usage
```python
from datasets import load_dataset
ds = load_dataset("HuggingFaceFW/dclm_100BT", split="train", streaming=True)
for sample in ds:
print(sample["text"][:200])
break
```
## Citation
```bibtex
@misc{niklaus2026smoldata,
title={SmolData},
author={Joel Niklaus and Hynek Kydl{\'\i}{\v{c}}ek},
year={2026},
publisher={Hugging Face},
journal={Hugging Face repository},
howpublished={\url{https://huggingface.co/collections/HuggingFaceFW/smol-data}}
}
```
提供机构：
maas
创建时间：
2026-02-16
