Open Source Indicators Project

The goal of the Open Source Indicators (OSI) Program was to make automated predictions of significant societal events through the continuous and automated analysis of publicly available data such as news media, social media, informational websites, and satellite imagery. Societal events of interest included civil unrest, disease outbreaks, and election results. Geographic areas of interest include countries in Latin America (LA) and the Middle East and North Africa (MENA). The handbook is intended to serve as a reference document for the OSI Program and a companion to the ground truth event data used for test and evaluation. The handbook provides guidance regarding the types of events considered; the submission of automated predictions or “warnings;” the development of ground truth; the test and evaluation of submitted warnings; performance measures; and other programmatic information. IARPA initiated a solicitation for OSI Research Teams in late summer 2011 for one base year and two option years of research. MITRE was selected as the Test and Evaluation (T&E) Team in November 2011. Following a review of proposals, three teams (BBN, HRL, and Virginia Tech (VT)) were selected. The OSI Program officially began in April 2012; manual event encoding and formal T&E ended in March 2015.

开源指标（Open Source Indicators, OSI）项目的目标是通过对新闻媒体、社交媒体、信息网站、卫星影像等公开可用数据开展持续自动化分析，实现对重大社会事件的自动化预测。该项目关注的社会事件包括民众骚乱、疾病暴发与选举结果，研究覆盖的地理区域为拉丁美洲（LA）、中东及北非（MENA）国家。本手册旨在作为OSI项目的参考文档，以及用于测试与评估的真实基准（ground truth）事件数据的配套指南。手册涵盖以下内容的相关规范：待评估事件类型、自动化预测或"预警"的提交流程、真实基准数据的构建、提交预警的测试与评估方法、性能评估指标，以及其他项目相关信息。美国情报高级研究计划局（IARPA）于2011年夏末发起OSI研究团队招募计划，提供1个基础研究年度与2个可选研究年度的研究资助。MITRE公司于2011年11月被选为测试与评估（T&E）团队；经提案评审后，最终选定BBN、HRL与弗吉尼亚理工大学（VT）共3支研究团队。OSI项目于2012年4月正式启动，人工事件编码与正式测试评估工作则于2015年3月结束。