allenai/dolmino-mix-1124

DOLMino Mix数据集是一个用于OLMo2模型第二阶段退火训练的混合数据集。它包含了多个高质量的数据源，如DCLM、Flan、Pes2o、Wiki、StackExchange等，涵盖了网页、STEM论文、百科全书、代码文本等多种类别。每个数据源的详细信息包括令牌数、未压缩字节数、文档数和许可证类型。数据集的混合比例根据不同的训练阶段（50B、100B、300B令牌）进行了调整，其中DCLM通常占50%的令牌产量。数据集的许可证为Open Data Commons Attribution License (ODC-By) v1.0。

The DOLMino Mix dataset is a mixture of high-quality data used for the second stage of OLMo2 model annealing training. It includes multiple high-quality data sources such as DCLM, Flan, Pes2o, Wiki, StackExchange, etc., covering various categories like web pages, STEM papers, encyclopedias, code text, and more. Detailed information for each data source includes the number of tokens, uncompressed bytes, number of documents, and license type. The mixture ratio of the dataset is adjusted according to different training stages (50B, 100B, 300B tokens), with DCLM typically accounting for 50% of the token yield. The dataset is licensed under the Open Data Commons Attribution License (ODC-By) v1.0.