ruliad/fineweb_350BT_chunk_35

---
dataset_info:
features:
- name: text
dtype: string
- name: id
dtype: string
- name: dump
dtype: string
- name: url
dtype: string
- name: date
dtype: string
- name: file_path
dtype: string
- name: language
dtype: string
- name: language_score
dtype: float64
- name: token_count
dtype: int64
splits:
- name: train
num_bytes: 17145771834
num_examples: 5000000
download_size: 10250516625
dataset_size: 17145771834
configs:
- config_name: default
data_files:
- split: train
path: data/train-*
---

The dataset includes various features such as text content, unique identifiers, URLs of data sources, recording dates, file paths, language types, language recognition scores, and token count statistics. The dataset is divided into a training set containing 5 million samples, with a total size of approximately 17.15GB and a download size of 10.25GB.

原始信息汇总

数据集信息

特征

• text: 类型为字符串
• id: 类型为字符串
• dump: 类型为字符串
• url: 类型为字符串
• date: 类型为字符串
• file_path: 类型为字符串
• language: 类型为字符串
• language_score: 类型为浮点数 (float64)
• token_count: 类型为整数 (int64)

分割

• train:
  • 字节数: 17145771834
  • 样本数: 5000000

大小

• 下载大小: 10250516625 字节
• 数据集大小: 17145771834 字节

配置

• config_name: default
  • data_files:
    • split: train
    • path: data/train-*