ruliad/fineweb_350BT_chunk_10

---
dataset_info:
features:
- name: text
dtype: string
- name: id
dtype: string
- name: dump
dtype: string
- name: url
dtype: string
- name: date
dtype: string
- name: file_path
dtype: string
- name: language
dtype: string
- name: language_score
dtype: float64
- name: token_count
dtype: int64
splits:
- name: train
num_bytes: 17059953514
num_examples: 5000000
download_size: 10215229671
dataset_size: 17059953514
configs:
- config_name: default
data_files:
- split: train
path: data/train-*
---

The dataset includes various features such as text content, unique identifiers, source URLs, record dates, file storage paths, text language, language detection scores, and token counts. The dataset is divided into a training set containing 5 million samples. The total size of the dataset is 17059953514 bytes, with a download size of 10215229671 bytes.

原始信息汇总

数据集概述

数据集特征

• text：字符串类型
• id：字符串类型
• dump：字符串类型
• url：字符串类型
• date：字符串类型
• file_path：字符串类型
• language：字符串类型
• language_score：浮点数类型
• token_count：整数类型

数据集分割

• train：
  • 字节数：17059953514
  • 示例数：5000000

数据集大小

• 下载大小：10215229671
• 数据集大小：17059953514

配置

• config_name: default
  • data_files:
    • split: train
      • path: data/train-*