timpal0l/line-extracted-full

该数据集包含两个主要特征：input_ids和attention_mask，分别以int32和int8的序列形式存储。数据集被分为训练集和测试集，训练集包含11,070,720个样本，测试集包含1,230,081个样本。数据集的下载大小为44,320,963,424字节，总大小为126,058,608,648字节。

This dataset includes two main features: input_ids and attention_mask, stored as sequences of int32 and int8 respectively. The dataset is divided into a training set and a test set, with the training set containing 11,070,720 samples and the test set containing 1,230,081 samples. The download size of the dataset is 44,320,963,424 bytes, and the total size is 126,058,608,648 bytes.

原始信息汇总

数据集概述

数据特征

• input_ids: 序列类型为 int32
• attention_mask: 序列类型为 int8

数据分割

• train:
  • 字节数: 113452738560.0
  • 样本数: 11070720
• test:
  • 字节数: 12605870088.0
  • 样本数: 1230081

数据大小

• 下载大小: 44320963424
• 数据集总大小: 126058608648.0

配置

• config_name: default
  • data_files:
    • train: data/train-*
    • test: data/test-*