spsither/tibetan_monolingual_S_cleaned_train_test_tokenized

---
dataset_info:
features:
- name: input_ids
sequence: int32
- name: attention_mask
sequence: int8
splits:
- name: train
num_bytes: 234340261728
num_examples: 91253996
- name: test
num_bytes: 12333698256
num_examples: 4802842
download_size: 5208492851
dataset_size: 246673959984
configs:
- config_name: default
data_files:
- split: train
path: data/train-*
- split: test
path: data/test-*
---

The dataset includes two main features: input_ids (input sequences) and attention_mask (attention masks). It is divided into a training set and a test set, with 91,253,996 samples in the training set and 4,802,842 samples in the test set. The download size of the dataset is 5,208,492,851 bytes, and the total size is 246,673,959,984 bytes. The dataset configuration is set to default, with the training and test data files stored in data/train-* and data/test-* paths respectively.

原始信息汇总

数据集概述

数据集特征

• input_ids：整数序列，数据类型为int32。
• attention_mask：整数序列，数据类型为int8。

数据集划分

• 训练集（train）：
  • 数据量：234340261728 字节
  • 样本数：91253996
• 测试集（test）：
  • 数据量：12333698256 字节
  • 样本数：4802842

数据集大小

• 下载大小：5208492851 字节
• 总数据集大小：246673959984 字节

数据文件配置

• 默认配置（default）：
  • 训练集路径：data/train-*
  • 测试集路径：data/test-*