Beetle-Data/ar-100M-pretok
收藏官方服务：
资源简介：
这是一个名为Beetle-Data/ar-100M-pretok的数据集，包含预分块的文本数据，每个数据块由513个标记的打包序列（input_ids）组成，确保无跨文档混合。数据集以分片形式增量上传，并在所有部分上传完成后通过data/_finalized.json文件标记为最终化。
This is a dataset named Beetle-Data/ar-100M-pretok consisting of pretokenized chunks, where each chunk is a packed sequence of 513 tokens (input_ids) with no cross-document bleeding. It is sharded incrementally, and the marker data/_finalized.json is committed once all parts are uploaded.
提供机构：
Beetle-Data
