joey00072/pretokenized__HuggingFaceFW_fineweb-edu__EleutherAI__gpt-neo-125m
收藏Hugging Face2024-11-22 更新2024-12-14 收录
下载链接：
https://hf-mirror.com/datasets/joey00072/pretokenized__HuggingFaceFW_fineweb-edu__EleutherAI__gpt-neo-125m
下载链接
链接失效反馈官方服务：
资源简介：
该数据集包含多个特征，包括id、dump、url、file_path、language、language_score、token_count、score、int_score、input_ids和attention_mask。数据集分为一个训练集，包含5,765,432个样本，总大小为24,942,837,662.25636字节，下载大小为13,644,753,644字节。数据集的配置名为default，数据文件路径为data/train-*。
The dataset contains multiple features including id, dump, url, file_path, language, language_score, token_count, score, int_score, input_ids, and attention_mask. The dataset is divided into one training set containing 5,765,432 samples, with a total size of 24,942,837,662.25636 bytes and a download size of 13,644,753,644 bytes. The datasets configuration name is default, and the data file path is data/train-*.
提供机构：
joey00072
