spsither/tibetan_monolingual_S_cleaned_train_test_tokenized

--- dataset_info: features: - name: input_ids sequence: int32 - name: attention_mask sequence: int8 splits: - name: train num_bytes: 234340261728 num_examples: 91253996 - name: test num_bytes: 12333698256 num_examples: 4802842 download_size: 5208492851 dataset_size: 246673959984 configs: - config_name: default data_files: - split: train path: data/train-* - split: test path: data/test-* ---The dataset includes two main features: input_ids (input sequences) and attention_mask (attention masks). It is divided into a training set and a test set, with 91,253,996 samples in the training set and 4,802,842 samples in the test set. The download size of the dataset is 5,208,492,851 bytes, and the total size is 246,673,959,984 bytes. The dataset configuration is set to default, with the training and test data files stored in data/train-* and data/test-* paths respectively.

原始信息汇总

数据集概述

数据集特征

• input_ids：整数序列，数据类型为int32。
• attention_mask：整数序列，数据类型为int8。

数据集划分

• 训练集（train）：
  • 数据量：234340261728 字节
  • 样本数：91253996
• 测试集（test）：
  • 数据量：12333698256 字节
  • 样本数：4802842

数据集大小

• 下载大小：5208492851 字节
• 总数据集大小：246673959984 字节

数据文件配置

• 默认配置（default）：
  • 训练集路径：data/train-*
  • 测试集路径：data/test-*