moca-embed/dclm_20b
收藏Hugging Face2025-07-01 更新2025-10-25 收录
下载链接：
https://hf-mirror.com/datasets/moca-embed/dclm_20b
下载链接
链接失效反馈官方服务：
资源简介：
这是一个用于MoCa模型模态感知连续预训练的文本预训练数据集。该数据集基于DCLM数据集改编，并随机下采样到大约200亿个token。数据集包含文本示例，其中`text`字段包含文本内容，而`images`字段为空，因为该数据集中不包含图像。
This is a text pre-training dataset used for modality-aware continual pre-training of MoCa models. It is adapted from DCLM and randomly downsampled to ~20B tokens. The dataset consists of text examples, where the `text` field contains the text content, and the `images` field is left blank as no images are included in the dataset.
提供机构：
moca-embed
