yourbench/highpassfilter-medical-documents-o4-mini

这是一个文本数据集，包含了文本的id、内容、元数据等信息。元数据中包含文本的来源URL、域名、Warc相关信息等。此外，还包含了对文本质量的一系列评估指标，如重复词汇的比例、句子数量、熵值等。数据集还包含了文本的PID、响应内容、医学相关分数和推理信息。数据集分为训练集，可用于文本质量评估和医学相关任务。

This is a text dataset that includes the id, content, metadata, and other information of the text. The metadata contains the source URL, domain, Warc-related information, etc. In addition, it includes a series of indicators for evaluating text quality, such as the proportion of duplicate words, the number of sentences, entropy, etc. The dataset also includes the PID of the text, response content, medical-related scores, and reasoning information. The dataset is divided into a training set, which can be used for text quality assessment and medical-related tasks.