Event Consequence Collections from Wikinews (ECCW)

Goodness of causal knowledge “extraction” systems are typically determined by performance over some benchmarks and although quite a few of them already (BECAUSE, SemEval) exist, they’re marred by the following problems: a) focusing mainly on common-sense reasoning tasks b) making restricting assumptions, e.g. causes and effects being word tokens, or trigger verbs etc. <br> <br> This endeavor seeks to build new benchmarks for causal knowledge extraction which can address at least some limitations of the previous benchmarks, particularly geared towards risk management and event forecasting. This dataset born of such an endeavor, which is also a result of a collaboration between the International Business Machines Corporation (IBM) and Rensselaer Polytechnic Institute (RPI), seeks to focus on event forecasting tasks and derives solely from Wikinews. Out of all (10k+) category pages that exist on wikinews, various automated and manual filtering approaches were applied to vet them to a lesser number, then finally combined into 39 significant "event" / category pages. From all these event / category pages, the first / earliest news article is considered the source event and all the following ones are considered consequences. Various negative or non-consequences are also found, these are events which might be topically (or semantically related to original topic of source event or from related set of categories of source event / original category page) related to source event which can serve as related but eventually "non-consequences" for a given source event. Using such a dataset, one can hope to create benchmarks for event forecasting systems or even use this as a benchmark itself. Due to the volume and the sheer number of event-consequence-negative_example subsets possible, this dataset can also be used to create training and testing sets for supervised classifiers which could perform answer simple multiple choice questions geared towards event forecasting, e.g. one such task can be:<br> <br> Input Event: Massive anti-government protests in Egypt continue into second day, several killed<br> Choices: Hosni Mubarak steps down as president of Egypt Bomb threat on UK–Egypt plane; diverted to Greece Benin, Nigeria join African Union continental free trade bloc Gaza Strip reports first swine flu cases Answer: (1)<br> <br> or it can also be a question answering task of the form (one correct + one incorrect options): Input Event: Massive anti-government protests in Egypt continue into second day, several killed<br> Choices: Hosni Mubarak steps down as president of Egypt Bomb threat on UK–Egypt plane; diverted to Greece Answer: (1) Included within is the end result of mining almost all of wikinews into a small, succinct and concise dataset filled with significant events and their consequences and some non consequences. Every event, consequence and non-consequence has a number of fields, including but not limited to:<br> <br> a) categories<br> b) category_links<br> c) category_name<br> d) content<br> e) date<br> f) title<br> g) url for every wikinews article present. There may be other differing extraction specific metadata fields included as well depending on which category of article (source event, consequences, negative examples / non-consequences) is being considered. There are 39 lines / category pages in the JSONL file with the following being the complete summary for the same: <pre><code class="language-json">{"information": "Complete Summary", "total_consequences": 570.0, "mean_consequences": 14.615384615384615, "median_consequences": 6.0, "total_negatives": 780.0, "total_negatives_before": 390.0, "total_negatives_after": 390.0}</code></pre> <br> Please feel free to contact the following people with any questions or comments: <strong>Oktie Hassanzadeh</strong><br> hassanzadeh at us.ibm.com <strong>Gaurav Dass</strong><br> dassg2 at rpi.edu<br> dassgaurav93 at gmail.com因果知识“抽取”系统的优劣通常通过相关基准测试的表现来评估。尽管目前已有不少此类基准（如BECAUSE、SemEval），但它们均存在以下局限：a) 主要聚焦于常识推理任务；b) 存在诸多限制性假设，例如假设因果关系的两端均为词Token（Token），或仅以触发动词作为因果标识等。 本研究旨在构建新型的因果知识抽取基准，以弥补现有基准的部分不足，尤其面向风险管理与事件预测场景。本数据集正是该项研究的成果之一，由国际商业机器公司（International Business Machines Corporation, IBM）与伦斯勒理工学院（Rensselaer Polytechnic Institute, RPI）联合打造，专注于事件预测任务，且全部数据源自维基新闻（Wikinews）。 研究人员从维基新闻的1万余个分类页面中，通过自动化与人工结合的多轮筛选，将其缩减至39个具有代表性的“事件”/分类页面。以每个事件/分类页面中的首篇（最早的）新闻报道作为源事件，后续所有报道均视为该源事件的后续结果。此外，研究人员还收集了各类负样本/非后续结果：即与源事件主题（或语义相关、或属于源事件关联分类集合）相关，但实际上并非该源事件必然后续结果的事件。 依托该数据集，研究者可构建面向事件预测系统的基准测试，或直接将其作为基准数据集使用。由于该数据集可生成海量的事件-后续结果-负样本子集，其同样可用于生成监督分类器的训练与测试集，以完成面向事件预测的简单多项选择任务，例如： 示例任务1： 输入事件：埃及大规模反政府抗议活动持续至次日，已造成数人死亡 选项： 1. 穆巴拉克辞去埃及总统职务 2. 一架英埃航班遭遇炸弹威胁，迫降希腊 3. 贝宁与尼日利亚加入非洲大陆自由贸易区 4. 加沙地区报告首例猪流感病例 答案：(1) 或者也可采用问答式任务（包含一个正确选项与一个错误选项）： 输入事件：埃及大规模反政府抗议活动持续至次日，已造成数人死亡 选项： 1. 穆巴拉克辞去埃及总统职务 2. 一架英埃航班遭遇炸弹威胁，迫降希腊 答案：(1) 本数据集是对维基新闻全量内容进行挖掘、提炼后的精简集合，涵盖大量典型事件及其后续结果与非后续结果。每个事件、后续结果及非后续结果均包含若干字段，包括但不限于： a) 分类标签 b) 分类链接 c) 分类名称 d) 内容正文 e) 发布日期 f) 标题 g) 对应维基新闻文章的URL 根据样本所属类别（源事件、后续结果、负样本/非后续结果）的不同，还可能包含其他与抽取任务相关的元数据字段。 该JSONL格式的数据文件共包含39条分类页面数据，其完整统计信息如下： <pre><code class="language-json">{"information": "Complete Summary", "total_consequences": 570.0, "mean_consequences": 14.615384615384615, "median_consequences": 6.0, "total_negatives": 780.0, "total_negatives_before": 390.0, "total_negatives_after": 390.0}</code></pre> 如有任何疑问或建议，请联系以下人员： 奥克蒂·哈萨扎德（Oktie Hassanzadeh） 邮箱：[e-mail] 高拉夫·达斯（Gaurav Dass） 邮箱：[e-mail]、[e-mail]