sboughorbel/redpajama-data-1b-tokenized-pythia-1b-deduped

该数据集包含一个名为train的训练分割，共有930,514个样本，总大小为3,815,107,400字节。数据集的特征包括一个名为input_ids的序列，其数据类型为int32。数据集的下载大小为1,156,889,066字节。

This dataset includes a training split named train with 930,514 examples and a total size of 3,815,107,400 bytes. The features of the dataset include a sequence named input_ids with a data type of int32. The download size of the dataset is 1,156,889,066 bytes.