ray0rf1re/Fineweb-Tiny

---
license: odc-by
language:
- en
size_categories:
- 100B<n<1T
---
# Fineweb-Tiny

## Dataset Description

**Fineweb-Tiny** is a highly curated, premium subset extracted from [`nampdn-ai/mini-fineweb`](https://huggingface.co/datasets/nampdn-ai/mini-fineweb).

### How "The Best" Was Determined
This dataset was created programmatically by streaming the original dataset and sorting chunks based on a rigorous quality scoring algorithm. The heuristic heavily favors:
1. High `language_score` (if provided by the upstream extraction).
2. Optimal document length (penalizing abnormally short snippets and excessively long, unformatted dumps).
3. Strong structural coherence suitable for pre-training Small Language Models (SLMs) and LLMs.

**Fineweb-Tiny** contains the absolute **BEST 72.9 Gigabytes** (compressed Parquet) of the original source. It drops the bottom 50% of lower-quality data from the stream to ensure dense, high-utility text.

### License
This dataset is released under the **Open Data Commons Attribution License (ODC-By) v1.0** to perfectly match the upstream Fineweb mini licensing constraints.