aklein4/mixed-pretraining-tokenized
收藏Hugging Face2025-07-21 更新2025-10-25 收录
下载链接：
https://hf-mirror.com/datasets/aklein4/mixed-pretraining-tokenized
下载链接
链接失效反馈官方服务：
资源简介：
该数据集包含文本输入和输出，适用于机器翻译或文本生成任务。数据集被划分为训练集，其大小为约52.37 GB，共有约4321.44万个示例。数据集以uint16类型的列表形式存储输入和输出的token ID，并记录了每个输入和输出的token数量。
The dataset consists of text input and output, suitable for machine translation or text generation tasks. The dataset is split into a training set, which is approximately 52.37 GB in size and contains about 43.21 million examples. The dataset stores input and output token IDs in the form of uint16 lists and records the number of tokens for each input and output.
提供机构：
aklein4
