WuDaoCorpora-Text

WuDaoCorpora是北京智源人工智能研究院（智源研究院）构建的大规模、高质量数据集，用于支撑大模型训练研究。目前由文本、对话、图文对、视频文本对四部分组成，分别致力于构建微型语言世界、提炼对话核心规律、打破图文模态壁垒、建立视频文字关联，为大模型训练提供坚实的数据支撑。

WuDaoCorpora is a large-scale, high-quality dataset developed by Beijing Academy of Artificial Intelligence (BAAI) to support research on large model training. Currently, it comprises four components: text corpora, dialogue datasets, image-text pairs, and video-text pairs. These components are respectively dedicated to constructing a miniature linguistic world, distilling core regularities of human dialogue, breaking the modality barriers between image and text modalities, and establishing the correlation between video and text, thereby providing solid data support for large model training.

搜集汇总

背景与挑战

背景概述

WuDaoCorpora-Text是北京智源人工智能研究院构建的大规模文本数据集，用于支撑大模型训练研究。该数据集通过严格清洗从100TB原始网页数据中提取，注重隐私保护，包含教育、科技等50多个行业标签，总规模达5TB，其中开源200GB数据以JSON格式提供，支持多领域预训练。