zehnlab/tajik-corpora-1.5Btokens

该数据集包含多个字段，如唯一标识符、数据集ID、数据集名称、源文件路径、源文件键、内容MD5哈希值以及内容本身。内容字段包含了文本数据，同时记录了文本的字符长度。数据集分为训练集、验证集和开发集，分别用于模型训练和评估。训练集包含349652个示例，验证集包含5351个示例，开发集包含1785个示例。数据集的总下载大小为约2.98GB，总存储大小为约6.87GB。

The dataset contains multiple fields such as unique identifiers, dataset IDs, dataset names, source file paths, source file keys, content MD5 hash values, and the content itself. The content field includes text data, and the length of the characters in the text is recorded. The dataset is split into training, validation, and development sets for model training and evaluation. The training set contains 349,652 examples, the validation set contains 5,351 examples, and the development set contains 1,785 examples. The total download size of the dataset is approximately 2.98GB, and the total storage size is approximately 6.87GB.