SmallDoge/SmallCorpus

SmallCorpus是一个大规模的多语言数据集，旨在用于训练小型语言模型。它包含多种类型的文本，包括网页内容、教育教材、编程代码、数学问题和链式思维推理示例。该数据集的结构旨在促进跨多种语言的语言模型预训练和持续预训练。

SmallCorpus is a large-scale multilingual dataset designed for training small language models. It includes various types of texts such as web content, educational textbooks, programming code, mathematical problems, and chain-of-thought reasoning examples. The structure of the dataset is intended to facilitate pre-training and continued pre-training of language models across multiple languages.