claran/seed-pretrain-decon

该数据集包含两个主要配置：gutenberg和wikipedia。gutenberg配置包含与古腾堡项目相关的书籍数据，wikipedia配置包含维基百科的文章数据。数据集主要用于文本生成任务，语言为英语。该数据集是用于预训练种子模型的语料库，将在EMNLP 2024会议上展示。

The dataset contains two main configurations: gutenberg and wikipedia. The gutenberg configuration includes book data related to the Gutenberg project, while the wikipedia configuration includes article data from Wikipedia. The dataset is primarily used for text generation tasks and is in English. This dataset serves as a pre-training corpus for seed models and will be presented at EMNLP 2024.