person65/fineweb-plus-fineweb-edu-10BT

---
license: apache-2.0
task_categories:
- text-generation
language:
- en
pretty_name: fineweb and fineweb edu 10bt samples combined
size_categories:
- 10B<n<100B
---
###### FineWeb 10BT+FineWeb-EDU 10BT combined
This dataset is a combination of the 10BT Fineweb samples and 10BT fineweb EDU
samples in a 3:7 ratio. There is about 14BT in total. (it says 10M to 100M for some reason.)\
It is **also deduplicated**, so it will not cause overfitting.

How to download it:
```python
from datasets import load_dataset

dataset = load_dataset("person65/fineweb-plus-fineweb-edu-10BT")
```
If you want to train a small model with a **decent amount of educational and general knowledge**, use this dataset.