fineweb_edu_100BT-shuffled

# FineWeb-Edu 100BT (Shuffled) A globally shuffled version of [HuggingFaceFW/fineweb_edu_100BT](https://huggingface.co/datasets/HuggingFaceFW/fineweb_edu_100BT). Part of the [Smol-Data](https://huggingface.co/collections/HuggingFaceFW/smol-data) collection — tried and tested mixes for strong pretraining. ## Dataset Description This dataset contains the same ~100B tokens as [fineweb_edu_100BT](https://huggingface.co/datasets/HuggingFaceFW/fineweb_edu_100BT) but with all documents globally shuffled (seed=42). Use this version when you need randomized document ordering for pretraining. ## How It Was Created The unshuffled dataset was loaded into memory, shuffled with `dataset.shuffle(seed=42)`, and re-uploaded with 100 shards. See the [smol_data.py](https://github.com/huggingface/datatrove/blob/main/examples/smol_data.py) script for details. ## Usage ```python from datasets import load_dataset ds = load_dataset("HuggingFaceFW/fineweb_edu_100BT-shuffled", split="train", streaming=True) for sample in ds: print(sample["text"][:200]) break ``` ## Citation ```bibtex @misc{niklaus2026smoldata, title={SmolData}, author={Joel Niklaus and Hynek Kydl{\'\i}{\v{c}}ek}, year={2026}, publisher={Hugging Face}, journal={Hugging Face repository}, howpublished={\url{https://huggingface.co/collections/HuggingFaceFW/smol-data}} } ```