Beetle-Data/he-100M-pretok
收藏官方服务：
资源简介：
预分词块（输入ID为513个标记的打包序列，无跨文档泄露）。数据分片逐步上传；标记文件`data/_finalized.json`在所有部分上传完成后提交。
Pre-tokenized chunks: packed sequences with 513 tokens as input IDs, without cross-document leakage. Data shards will be uploaded incrementally; the manifest file `data/_finalized.json` must be submitted after all parts have finished uploading.
提供机构：
Beetle-Data
