Beetle-Data/et-2B-pretok
收藏官方服务：
资源简介：
Beetle-Data/et-2B-pretok是一个预分块的数据集，每个块由513个标记的打包序列组成，确保没有跨文档的混合。数据集以分片形式提供，所有部分上传完成后会通过`data/_finalized.json`文件标记完成状态。
Beetle-Data/et-2B-pretok is a pretokenized dataset consisting of chunks with 513-token packed sequences and no cross-document bleeding. It is sharded incrementally, and the completion of all parts is marked by the `data/_finalized.json` file.
提供机构：
Beetle-Data
