williambrach/html-description-content

这个数据集包含了一组网页，每个网页都配有了完整的原始HTML内容和相应的真实纯文本内容。数据集的一个特点是添加了一个由LLM生成的（合成的）查询列，这个查询是一个简短的（1-2句）页面内容描述，旨在作为信息提取系统的提示或查询。数据集支持信息提取和HTML到文本转换的任务。数据集的字段包括页面ID、来源、HTML内容、查询和文本。数据集分为训练集和测试集，从不同的数据源中进行分层抽样。

This dataset contains a collection of web pages, each paired with full raw HTML content and its corresponding ground-truth plaintext content. A key feature of this dataset is the addition of a LLM-generated (synthetic) query column, which is a short (1-2 sentence) description of the pages content, designed to be used as a prompt or query for information extraction systems. The dataset supports tasks of information extraction and HTML-to-text conversion. The data fields include page_id, source, html, query, and text. The dataset is split into training and testing sets, with stratified sampling from different web sources.