Beetle-Data/no-100M-pretok
收藏官方服务：
资源简介：
这是一个预分词数据集，包含分块的513个token打包序列，无跨文档内容混合。数据以parquet格式存储，分片上传，并通过标记文件指示所有部分已上传完成。
This is a pretokenized dataset consisting of chunked 513-token packed sequences with no cross-document bleeding. The data is stored in parquet format, sharded incrementally, and a marker file indicates that all parts have been uploaded.
提供机构：
Beetle-Data
