likhith45688/lm_dataset

---
configs:
- config_name: default
data_files:
- split: train
path: data/train-*
- split: valid
path: data/valid-*
dataset_info:
features:
- name: input_ids
sequence: int32
- name: attention_mask
sequence: int8
- name: labels
sequence: int64
splits:
- name: train
num_bytes: 606341604
num_examples: 361779
- name: valid
num_bytes: 144454440
num_examples: 86190
download_size: 137305987
dataset_size: 750796044
---
# Dataset Card for "lm_dataset"

[More Information needed](https://github.com/huggingface/datasets/blob/main/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)

原始信息汇总

数据集概述

配置

• 默认配置 (default)
  • 数据文件
    • 训练集 (train): data/train-*
    • 验证集 (valid): data/valid-*

数据集信息

• 特征
  • input_ids: 序列类型为 int32
  • attention_mask: 序列类型为 int8
  • labels: 序列类型为 int64
• 数据分割
  • 训练集 (train)
    • 字节数: 606341604
    • 样本数: 361779
  • 验证集 (valid)
    • 字节数: 144454440
    • 样本数: 86190
• 下载大小: 137305987
• 数据集大小: 750796044