kz919/pg19-llama3-tokenized

该数据集包含两个特征字段：input_ids和attention_mask，分别存储为int32和int8类型的序列。数据集仅包含训练集split，共有28602个样本，总大小为14095077421字节。具体的数据集内容描述在README中未提供。

The dataset includes two feature fields: input_ids and attention_mask, stored as int32 and int8 sequences respectively. The dataset contains only a training set split with a total of 28,602 samples and a size of 14,095,077,421 bytes. Specific content description of the dataset is not provided in the README.

原始信息汇总

数据集概述

特征

• input_ids: 序列类型为 int32
• attention_mask: 序列类型为 int8

数据分割

• train:
  • 字节数: 14095077421
  • 样本数: 28602

数据集大小

• 下载大小: 4937829298 字节
• 数据集大小: 14095077421 字节

配置

• config_name: default
  • data_files:
    • split: train
    • path: data/train-*