allenai/OLMoE-mix-0924
收藏Hugging Face2024-12-02 更新2024-12-14 收录
下载链接：
https://hf-mirror.com/datasets/allenai/OLMoE-mix-0924
下载链接
链接失效反馈官方服务：
资源简介：
OLMoE Mix数据集用于训练OLMoE-1B-7B模型，这是一个包含1B活跃参数和7B总参数的混合专家语言模型。数据集包含多个子集，如DCLM Baseline 1.0、Starcoder、peS2o等，并提供了每个子集的统计信息，如token数、单词数、字节数和文档数。数据预处理包括移除包含32个或更多重复ngrams的文档，并对Starcoder数据集进行了额外的处理。数据集采用Open Data Commons Attribution License (ODC-By) v1.0许可。
The OLMoE Mix (September 2024) dataset was used to train the OLMoE-1B-7B model, a Mixture-of-Experts language model with 1B active and 7B total parameters. The dataset consists of multiple subsets, each with detailed statistics such as the number of tokens, words, bytes, and documents. Preprocessing steps include removing repeated ngrams and documents under specific conditions. The dataset is licensed under the Open Data Commons Attribution License (ODC-By) v1.0.
提供机构：
allenai
