HCAI-Lab/dolma3-olmo3-corpus-manifest

dolma3-olmo3-corpus-manifest 是一个统一按文档构建的清单变体数据集，基于OLMo3辅助模式，包含11亿文档，采用32列PyArrow模式，涵盖主题、格式、质量和令牌计数以及源分片信息。dolma3-olmo3-corpus-manifest is a manifest-style variant dataset constructed uniformly at the document level, based on the OLMo3 auxiliary mode. It contains 1.1 billion documents, adopts a 32-column PyArrow schema, and covers metadata including topic, format, quality, token count, and source shard information.