Alignment-Lab-AI/wikitext-2-raw-bytepair-standard

该数据集包含三个主要部分：特征、数据分割和配置。特征部分定义了一个名为bytepair_ids的序列，其数据类型为int64。数据分割部分包括测试集、训练集和验证集，分别包含2891、23767和2461个样本，对应的字节大小分别为5156396.0、43707492.0和4581844.0字节。配置部分指定了每个分割对应的数据文件路径。

The dataset consists of three main parts: features, splits, and configs. The features section defines a sequence named bytepair_ids with a data type of int64. The splits section includes test, train, and validation sets, containing 2891, 23767, and 2461 examples respectively, with corresponding byte sizes of 5156396.0, 43707492.0, and 4581844.0 bytes. The configs section specifies the data file paths for each split.