SlimPajama-627B
收藏OpenXLab2026-04-18 收录
官方服务：
资源简介：
The latest research has shown that data quality is as important as data quantity. While training on more than one data epoch can be beneficial, this should be a choice rather than a side-effect of duplicates in the dataset. We decided to extensively deduplicate RedPajama to produce a dataset with higher information density. This means when using SlimPajama, you can achieve higher accuracy with the same compute budget when compared to other datasets.
提供机构：
OpenDataLab创建时间：
2023-10-11
