haritzpuerto/the_pile_00_OpenWebText2

该数据集包含文本数据和元数据，其中元数据包括pile_set_name字段。数据集分为训练集、验证集和测试集，分别包含1079800、33400和32925个样本。数据集的下载大小为2781879990字节，总大小为4527665769字节。数据文件路径分别为data/train-*、data/validation-*和data/test-*。The dataset contains text data and metadata, where the metadata includes the pile_set_name field. The dataset is divided into training, validation, and test sets, containing 1079800, 33400, and 32925 samples respectively. The download size of the dataset is 2781879990 bytes, and the total size is 4527665769 bytes. The data file paths are data/train-*, data/validation-*, and data/test-*.