Tony068/TinyStories-tokenized-falcon-7B-tokenizer
收藏Hugging Face2025-02-15 更新2025-04-12 收录
下载链接：
https://hf-mirror.com/datasets/Tony068/TinyStories-tokenized-falcon-7B-tokenizer
下载链接
链接失效反馈官方服务：
资源简介：
这是一个使用falcon-7B tokenizer处理的小故事数据集的tokenized版本，总共有10亿8529万6128个token。数据集包含文本内容、输入ID序列和注意力掩码三个字段，并划分为训练集和验证集。
This is the tokenized version of the roneneldan/TinyStories dataset, tokenized using the falcon-7B tokenizer. The dataset contains a total of one billion eight hundred and fifty-two million nine hundred and sixty-one thousand two hundred and eighty-eight tokens. It includes fields for text content, input ID sequences, and attention masks, and is split into training and validation sets.
提供机构：
Tony068
