dantedgp/cwa-1M-tokenized

该数据集包含了两个特征字段：input_ids和attention_mask，前者为int32类型的序列，后者为int8类型的序列。数据集分为训练集(train)和评估集(eval)，其中训练集包含835,900个示例，大小为1.7GB；评估集包含208,976个示例，大小为0.4GB。数据集的总大小为2.1GB，下载大小为270MB。

The dataset includes two feature fields: input_ids and attention_mask, with the former being a sequence of int32 type and the latter being a sequence of int8 type. The dataset is split into a training set (train) and an evaluation set (eval), with the training set containing 835,900 examples and sized at 1.7GB, while the evaluation set contains 208,976 examples and is 0.4GB in size. The total size of the dataset is 2.1GB, and the download size is 270MB.