ruliad/fineweb_350BT_chunk_39

---
dataset_info:
features:
- name: text
dtype: string
- name: id
dtype: string
- name: dump
dtype: string
- name: url
dtype: string
- name: date
dtype: string
- name: file_path
dtype: string
- name: language
dtype: string
- name: language_score
dtype: float64
- name: token_count
dtype: int64
splits:
- name: train
num_bytes: 17116271565
num_examples: 5000000
download_size: 10226939195
dataset_size: 17116271565
configs:
- config_name: default
data_files:
- split: train
path: data/train-*
---

The dataset includes various features such as text content, unique identifiers, URLs of data sources, record dates, file paths, language types, language identification confidence scores, and token count statistics of the text. The dataset is divided into a training set with a total of 5 million samples. The total size of the dataset is 17116271565 bytes, and the download size is 10226939195 bytes.

原始信息汇总

数据集概述

数据集特征

• text：字符串类型
• id：字符串类型
• dump：字符串类型
• url：字符串类型
• date：字符串类型
• file_path：字符串类型
• language：字符串类型
• language_score：浮点数类型
• token_count：整数类型

数据集分割

• train：
  • 字节数：17116271565
  • 示例数：5000000

数据集大小

• 下载大小：10226939195
• 数据集大小：17116271565

配置

• default：
  • 训练数据路径：data/train-*