ontocord/MixtureVitae-211BT
收藏官方服务：
资源简介：
MixtureVitae是一个由三个主要部分组成的2110亿 token的开放预训练数据集，包括精选源文本、指令与推理数据以及网络内容。该数据集注重使用宽松许可和低风险数据，旨在不依赖不受限制的网络抓取的情况下训练出有竞争力的大型语言模型。数据集的文档由特殊token分隔，适用于模型训练。
MixtureVitae is an open pretraining dataset with 211B tokens, composed of three major components: curated source text, instruction and reasoning data, and web content. The dataset focuses on using permissive-licensed and low-risk data to train competitive large language models without relying on unrestricted web scraping. The documents in the dataset are separated by special tokens, suitable for model training.
提供机构：
ontocord
