toksuitebackup/tokenmonster-englishcode-32000-consistent-v1-toksuite-detokenized
收藏Hugging Face2025-11-17 更新2025-12-20 收录
下载链接：
https://hf-mirror.com/datasets/toksuitebackup/tokenmonster-englishcode-32000-consistent-v1-toksuite-detokenized
下载链接
链接失效反馈官方服务：
资源简介：
---
language:
- en
- tr
- fa
- zh
- it
configs:
- config_name: chunk-0
data_files:
- split: train
path: train_data_0*-8.jsonl
- config_name: chunk-1
data_files:
- split: train
path: train_data_1*-8.jsonl
- config_name: chunk-2
data_files:
- split: train
path: train_data_2*-8.jsonl
- config_name: chunk-3
data_files:
- split: train
path: train_data_3*-8.jsonl
- config_name: chunk-4
data_files:
- split: train
path: train_data_4*-8.jsonl
- config_name: chunk-5
data_files:
- split: train
path: train_data_5*-8.jsonl
- config_name: chunk-6
data_files:
- split: train
path: train_data_6*-8.jsonl
- config_name: chunk-7
data_files:
- split: train
path: train_data_7*-8.jsonl
---
Training data of the model detokenized in the exact order seen by the model.
The training data is partitioned into **8 chunks** (`chunk-0` through `chunk-7`), based on the GPU rank that generated the data. Each chunk contains detokenized text files in JSON Lines format (`.jsonl`).
提供机构：
toksuitebackup
