ruliad/fineweb_edu_100BT_chunk_4

---
dataset_info:
features:
- name: text
dtype: string
- name: id
dtype: string
- name: dump
dtype: string
- name: url
dtype: string
- name: file_path
dtype: string
- name: language
dtype: string
- name: language_score
dtype: float64
- name: token_count
dtype: int64
- name: score
dtype: float64
- name: int_score
dtype: int64
splits:
- name: train
num_bytes: 25471960160
num_examples: 5000000
download_size: 14733006397
dataset_size: 25471960160
configs:
- config_name: default
data_files:
- split: train
path: data/train-*
---

The dataset includes multiple features such as text, ID, URL, file path, language, language score, token count, score, and integer score. The data types cover string, float, and integer. The dataset is divided into a training set with 5 million examples, with a total size of 25471960160 bytes and a download size of 14733006397 bytes.

原始信息汇总

数据集概述

数据集特征

• text：字符串类型
• id：字符串类型
• dump：字符串类型
• url：字符串类型
• file_path：字符串类型
• language：字符串类型
• language_score：浮点数类型
• token_count：整数类型
• score：浮点数类型
• int_score：整数类型

数据集划分

• train：
  • 示例数量：5000000
  • 数据大小：25471960160 字节

数据集大小

• 下载大小：14733006397 字节
• 数据集总大小：25471960160 字节

配置

• default 配置：
  • 训练数据路径：data/train-*