skymizer/fineweb-edu-dedup-45B-2-of-4
收藏Hugging Face2025-01-11 更新2025-02-15 收录
下载链接：
https://hf-mirror.com/datasets/skymizer/fineweb-edu-dedup-45B-2-of-4
下载链接
链接失效反馈官方服务：
资源简介：
这是一个包含文本数据的训练集，数据字段包括文本内容、唯一标识符和元数据。元数据中提供了dump信息、URL、日期、文件路径、语言类型、语言分数、token数量、分数和整数分数等详细信息。数据集大小约为55.98GB，共有约1084.21万个示例。
This dataset is a training set containing text data, with fields including text content, unique identifier, and metadata. The metadata provides detailed information such as dump information, URL, date, file path, language type, language score, token count, score, and integer score. The dataset is approximately 55.98GB in size and contains about 10.842 million examples.
提供机构：
skymizer
