alvations/annotated-wiki-2016

Annotated Wikipedia 2016是一个基于2016年英文维基百科快照的数据集，数据被处理为JSON格式，并保留了内联超链接作为实体链接标注。每个文章包含完整的纯文本以及一个标注列表，每个标注由(surface_form, target_uri, character_offset)元组组成，对应源文章中的每个维基链接。surface_form是链接在文章中出现的文本片段，uri是链接目标页面的维基百科URL格式（如Political_philosophy），offset是surface_form在文本中的字符偏移量（从零开始）。数据集未进行过滤、文本标准化或实体集限制，标注保持原始状态，包括重复链接和重定向目标。数据规模在1M到10M之间，适用于文本生成、标记分类和文本检索等NLP任务。English Wikipedia (~2016 snapshot) processed into JSON, with inline hyperlinks preserved as gold-aligned entity-link annotations. Each article carries its full plain text plus a list of (surface_form, target_uri, character_offset) tuples — one per wikilink in the source. The dataset includes annotations where surface_form is the literal text span as it appears in the article, uri is the linked target page in Wikipedia URL form, and offset is the zero-based character offset into text. No filtering, text normalisation, or entity-set restriction is applied, and annotations are kept verbatim including duplicate links and redirect targets. The size category is 1M<n<10M, and it is suitable for tasks such as text-generation, token-classification, and text-retrieval.