matdmiller/finemath-4plus

这是一个包含网页文本内容的数据集，分为不同的规模版本，包括100k、10k、1M、1k和完整的(full)版本。每个版本都包含训练集和验证集，适用于自然语言处理任务。数据集的特征包括网页的URL、抓取时间、MIME类型、Warc文件信息、文本内容、词汇和字符计数、元数据、得分信息、爬虫和快照类型、语言信息等。

This dataset contains web page text content, divided into different scale versions including 100k, 10k, 1M, 1k, and the full version. Each version includes a training set and a validation set, suitable for natural language processing tasks. The features of the dataset include the URL of the web page, fetch time, MIME type, Warc file information, text content, vocabulary and character count, metadata, score information, crawler and snapshot type, language information, etc.