raileymontalan/SEA-PILE-v2-tl-tokenized

该数据集包含多个字段，如dump（字符串类型）、时间戳（毫秒级时间戳）、url（字符串类型）、warc记录ID（字符串类型）、ids（整数列表）和长度（整数类型）。数据集分为训练集和验证集，其中训练集包含4538452个样本，大小为22378798572字节；验证集包含45843个样本，大小为224902251字节。数据集的总大小为22603700823字节，下载大小为4810849795字节。

The dataset includes multiple fields such as dump (string type), timestamp (millisecond timestamp), url (string type), warc-record-id (string type), ids (list of integers), and length (integer type). The dataset is split into a training set and a validation set, with the training set containing 4,538,452 samples and a size of 22,378,798,572 bytes; the validation set contains 45,843 samples and a size of 224,902,251 bytes. The total size of the dataset is 22,603,700,823 bytes, and the download size is 4,810,849,795 bytes.