skymizer/pretraining-5B-tokenized-padded
收藏Hugging Face2024-11-15 更新2024-12-14 收录
下载链接：
https://hf-mirror.com/datasets/skymizer/pretraining-5B-tokenized-padded
下载链接
链接失效反馈官方服务：
资源简介：
该数据集包含文本数据，每个样本包括文本内容、文本的token数量以及对应的输入ID序列。数据集分为训练集和测试集，训练集包含5,732,022个样本，测试集包含1,141个样本。总下载大小为21,407,549,071字节，数据集总大小为39,735,794,135字节。
This dataset contains text data, with each sample including the text content, the number of tokens in the text, and the corresponding input ID sequence. The dataset is divided into a training set and a test set, with the training set containing 5,732,022 samples and the test set containing 1,141 samples. The total download size is 21,407,549,071 bytes, and the total dataset size is 39,735,794,135 bytes.
提供机构：
skymizer
