ruliad/fineweb_350BT_chunk_66
收藏Hugging Face2024-06-02 更新2024-06-12 收录
下载链接：
https://hf-mirror.com/datasets/ruliad/fineweb_350BT_chunk_66
下载链接
链接失效反馈官方服务：
资源简介：
---
dataset_info:
features:
- name: text
dtype: string
- name: id
dtype: string
- name: dump
dtype: string
- name: url
dtype: string
- name: date
dtype: string
- name: file_path
dtype: string
- name: language
dtype: string
- name: language_score
dtype: float64
- name: token_count
dtype: int64
splits:
- name: train
num_bytes: 17290469073
num_examples: 5000000
download_size: 10319557721
dataset_size: 17290469073
configs:
- config_name: default
data_files:
- split: train
path: data/train-*
---
This dataset includes various features such as text content, unique identifiers, URLs of data sources, record dates, file paths, language types, language detection scores, and token counts of the text. The dataset is divided into a training set with 5,000,000 examples. The total size of the dataset is 17,290,469,073 bytes, with a download size of 10,319,557,721 bytes.
提供机构：
ruliad
原始信息汇总
数据集概述
数据集特征
- text: 数据类型为字符串
- id: 数据类型为字符串
- dump: 数据类型为字符串
- url: 数据类型为字符串
- date: 数据类型为字符串
- file_path: 数据类型为字符串
- language: 数据类型为字符串
- language_score: 数据类型为浮点数
- token_count: 数据类型为整数
数据集分割
- train:
- 字节数: 17290469073
- 示例数: 5000000
数据集大小
- 下载大小: 10319557721
- 数据集大小: 17290469073
配置
- config_name: default
- data_files:
- split: train
- path: data/train-*
