Yxanul/mixed-pretrain-3b
收藏Hugging Face2025-08-31 更新2025-10-25 收录
下载链接：
https://hf-mirror.com/datasets/Yxanul/mixed-pretrain-3b
下载链接
链接失效反馈官方服务：
资源简介：
混合预训练数据集3B是一个包含30亿个标记的精心策划和混合的预训练数据集，来源于高质量的教育、数学和编程资源。该数据集由60%的FineWeb-Edu（教育网页内容）、30%的数学问题和解答以及10%的Python代码组成。数据集以Parquet格式存储，使用cl100k_base标记器。它适用于语言模型的预训练、针对教育内容的微调、跨领域的多任务学习和课程学习研究。
The Mixed Pretraining Dataset 3B is a carefully curated and mixed pretraining dataset containing 3 billion tokens from high-quality educational, mathematical, and programming sources. It combines 60% FineWeb-Edu, 30% Mathematics, and 10% Python Code. The dataset is stored in Parquet format using the cl100k_base tokenizer and is designed for language model pretraining, fine-tuning on educational content, multi-task learning across domains, and research on curriculum learning.
提供机构：
Yxanul
