timaeus/dsir-pile-13m-filtered-for-wikipedia-en

该数据集包含三个主要特征：contents（内容，数据类型为字符串）、metadata（元数据，包含一个名为pile_set_name的字符串序列）和id（标识符，数据类型为int64）。数据集分为一个训练集（train），包含450,004个样本，总大小为715,603,447.2530501字节。下载大小为453,537,302字节，数据集总大小与训练集大小相同。配置信息显示数据文件路径为data/train-*。

The dataset contains three main features: contents (content, data type is string), metadata (metadata, containing a sequence of strings named pile_set_name), and id (identifier, data type is int64). The dataset is divided into a training set (train) containing 450,004 samples, with a total size of 715,603,447.2530501 bytes. The download size is 453,537,302 bytes, and the total dataset size is the same as the training set size. The configuration information shows that the data file path is data/train-*.