semran1/finewebedu-dedup-600k

这是一个包含文本数据的训练集，其中每个数据点包括文本内容、唯一标识符以及元数据。元数据中提供了文本的来源URL、创建日期、文件路径、语言类型、语言置信度、词汇计数、评分和整数评分等信息。数据集大小为约3.1GB，共有60万个示例。

This is a training dataset containing text data, with each data point including text content, a unique identifier, and metadata. The metadata provides information such as the texts source URL, creation date, file path, language type, language confidence, word count, score, and integer score. The dataset size is approximately 3.1GB, with a total of 600,000 examples.