gokulsrinivasagan/wiki_book_corpus_processed_gpt2_dataset_tiny

该数据集包含三个主要特征：input_ids（序列类型，int32）、attention_mask（序列类型，int8）和labels（序列类型，int64）。数据集分为训练集和验证集，训练集包含660,643个样本，占用8,802,407,332字节；验证集包含33,767个样本，占用449,911,508字节。整个数据集的下载大小为2,756,142,672字节，总大小为9,252,318,840字节。

The dataset contains three main features: input_ids (sequence type, int32), attention_mask (sequence type, int8), and labels (sequence type, int64). The dataset is divided into a training set and a validation set. The training set contains 660,643 samples, occupying 8,802,407,332 bytes; the validation set contains 33,767 samples, occupying 449,911,508 bytes. The total download size of the dataset is 2,756,142,672 bytes, and the total size is 9,252,318,840 bytes.

原始信息汇总

数据集概述

特征信息

• input_ids: 序列类型为 int32
• attention_mask: 序列类型为 int8
• labels: 序列类型为 int64

数据分割

• train:
  • 字节数: 8802407332
  • 样本数: 660643
• validation:
  • 字节数: 449911508
  • 样本数: 33767

数据集大小

• 下载大小: 2756142672 字节
• 总大小: 9252318840 字节

配置信息

• config_name: default
  • data_files:
    • train: data/train-*
    • validation: data/validation-*