Firefly 中文 Llama2 增量预训练数据集

该数据集为 Firefly-LLaMA2-Chinese 项目 的增量预训练数据，一共约 22GB 文本，主要包含 CLUE 、 ThucNews 、 CNews 、 COIG 、维基百科等开源数据集，以及研究团队收集的古诗词、散文、文言文等，数据分布如下图。

This dataset serves as the incremental pre-training data for the Firefly-LLaMA2-Chinese project, containing approximately 22 GB of text in total. Its primary sources include open-source datasets such as CLUE, ThucNews, CNews, COIG, and Wikipedia, alongside classical Chinese poetry, prose, and classical Chinese texts collected by the research team. The data distribution is illustrated in the figure below.

搜集汇总

背景与挑战

背景概述

该数据集是Firefly-LLaMA2-Chinese项目的增量预训练数据，总规模约22GB，内容整合了CLUE、ThucNews等开源文本资源，并补充了古诗词和文言文等多样化语料。