AbdallahhSaleh/billion_tokenized

该数据集是一个包含文本处理相关特征的训练集，具体包括input_ids，token_type_ids和attention_mask三个字段。数据集的总大小为约12.03GB，其中训练集大小为约12.03GB，包含大约1671.5万个样本。数据集适用于机器学习模型的训练，特别是用于自然语言处理任务。

This dataset is a training set containing features related to text processing, including input_ids, token_type_ids, and attention_mask. The total size of the dataset is approximately 12.03GB, with the training set being about 12.03GB, containing about 16.715 million samples. The dataset is suitable for machine learning model training, especially for natural language processing tasks.