Beetle-Data/en-for-no-100M-pretok
收藏官方服务：
资源简介：
Beetle-Data/en-for-no-100M-pretok数据集包含预分词块，这些块是513个令牌的打包序列，确保无跨文档泄漏。数据以分片形式增量上传，上传完成后通过一个标记文件指示。
The Beetle-Data/en-for-no-100M-pretok dataset consists of pretokenized chunks, which are 513-token packed sequences with no cross-document bleeding. The data is sharded and uploaded incrementally, with a marker file indicating completion once all parts are uploaded.
提供机构：
Beetle-Data
