Intel/WEC-Eng
收藏数据集概述
数据集名称
WEC-Eng
数据集描述
WEC-Eng是一个大规模的跨文档事件共指消解数据集，从英文维基百科中提取。
语言
英语
数据集加载
使用huggingface_hub库加载WEC-Eng文件的示例代码如下：
json from huggingface_hub import hf_hub_url, cached_download import json REPO_ID = "datasets/Intel/WEC-Eng" splits_files = ["Dev_Event_gold_mentions_validated.json", "Test_Event_gold_mentions_validated.json", "Train_Event_gold_mentions.json"] wec_eng = list() for split_file in splits_files: wec_eng.append(json.load(open(cached_download( hf_hub_url(REPO_ID, split_file)), "r")))
数据集结构
数据分割
- 最终版本的英文CD事件共指数据集
- 训练集 - Train_Event_gold_mentions.json
- 开发集 - Dev_Event_gold_mentions_validated.json
- 测试集 - Test_Event_gold_mentions_validated.json
| 训练 | 验证 | 测试 | |
|---|---|---|---|
| 集群 | 7,042 | 233 | 322 |
| 事件提及 | 40,529 | 1,250 | 1,893 |
- 非（集群内）控制版本的数据集（词汇多样性）
- 全部（实验性） - All_Event_gold_mentions_unfiltered.json
数据实例
json { "coref_chain": 2293469, "coref_link": "Family Values Tour 1998", "doc_id": "House of Pain", "mention_context": [ "From", "then", "on", ",", "the", "members", "continued", "their" ], "mention_head": "Tour", "mention_head_lemma": "Tour", "mention_head_pos": "PROPN", "mention_id": "108172", "mention_index": 1, "mention_ner": "UNK", "mention_type": 8, "predicted_coref_chain": null, "sent_id": 2, "tokens_number": [ 50, 51, 52, 53 ], "tokens_str": "Family Values Tour 1998", "topic_id": -1 }
数据字段
| 字段 | 值类型 | 值描述 |
|---|---|---|
| coref_chain | 数值 | 共指链/集群ID |
| coref_link | 字符串 | 共指链接维基百科页面/文章标题 |
| doc_id | 字符串 | 提及页面/文章标题 |
| mention_context | 列表[字符串] | 提及段落的分词（包括提及） |
| mention_head | 字符串 | 提及跨度头部标记 |
| mention_head_lemma | 字符串 | 提及跨度头部标记词元 |
| mention_head_pos | 字符串 | 提及跨度头部标记POS |
| mention_id | 字符串 | 提及ID |
| mention_index | 数值 | 提及在json文件中的索引 |
| mention_ner | 字符串 | 提及NER |
| tokens_number | 列表[数值] | 提及上下文中的标记ID |
| tokens_str | 字符串 | 提及跨度文本 |
| topic_id | 忽略 | 忽略 |
| mention_type | 忽略 | 忽略 |
| predicted_coref_chain | 忽略 | 忽略 |
| sent_id | 忽略 | 忽略 |
引用信息
@inproceedings{eirew-etal-2021-wec, title = "{WEC}: Deriving a Large-scale Cross-document Event Coreference dataset from {W}ikipedia", author = "Eirew, Alon and Cattan, Arie and Dagan, Ido", booktitle = "Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies", month = jun, year = "2021", address = "Online", publisher = "Association for Computational Linguistics", url = "https://aclanthology.org/2021.naacl-main.198", doi = "10.18653/v1/2021.naacl-main.198", pages = "2498--2510", abstract = "Cross-document event coreference resolution is a foundational task for NLP applications involving multi-text processing. However, existing corpora for this task are scarce and relatively small, while annotating only modest-size clusters of documents belonging to the same topic. To complement these resources and enhance future research, we present Wikipedia Event Coreference (WEC), an efficient methodology for gathering a large-scale dataset for cross-document event coreference from Wikipedia, where coreference links are not restricted within predefined topics. We apply this methodology to the English Wikipedia and extract our large-scale WEC-Eng dataset. Notably, our dataset creation method is generic and can be applied with relatively little effort to other Wikipedia languages. To set baseline results, we develop an algorithm that adapts components of state-of-the-art models for within-document coreference resolution to the cross-document setting. Our model is suitably efficient and outperforms previously published state-of-the-art results for the task.", }
许可证
数据集根据<a href="https://creativecommons.org/licenses/by-sa/3.0/deed.en_US">Creative Commons Attribution-ShareAlike 3.0 Unported License</a>提供。
