skymizer/fineweb-edu-dedup-45B-2-of-2

这是一个包含文本内容、唯一标识符和元数据信息的文本数据集，适用于机器学习模型训练。元数据中提供了关于文本的详细信息，如来源链接、日期、语言类型、语言评分、token数量、文本评分等。数据集仅包含训练集，示例数量庞大，适合大规模的文本分析任务。

This text dataset includes text content, unique identifiers, and metadata information, suitable for machine learning model training. The metadata provides detailed information about the text, such as source URL, date, language type, language score, token count, text score, etc. The dataset contains only a training set with a large number of examples, making it suitable for large-scale text analysis tasks.