Beetle-Data/en-for-da-100M-pretok
收藏官方服务：
资源简介：
预分词的块（input_ids = 513个标记的打包序列，无跨文档信息泄露）。数据被增量分片；标记文件data/_finalized.json在所有部分上传完成后提交。
Pre-tokenized chunks (packed sequences of 513 tokens stored as input_ids, with no cross-document information leakage). The dataset is incrementally sharded; the finalized token file data/_finalized.json will be submitted once all parts have been uploaded.
提供机构：
Beetle-Data
