HumAID: Human-Annotated Disaster Incidents Data from Twitter with Deep Learning Benchmarks

The HumAID Twitter dataset consists of several thousands of manually annotated tweets that have been collected during nineteen major natural disaster events including earthquakes, hurricanes, wildfires, and floods, which happened during 2016 to 2019 across different parts of the World. It is the largest social media dataset (~77K) for crisis informatics so far (for details please refer to our paper). The annotations consist of following humanitarian categories. Humanitarian categories Caution and advice Displaced people and evacuations Dont know cant judge Infrastructure and utility damage Injured or dead people Missing or found people Not humanitarian Other relevant information Requests or urgent needs Rescue volunteering or donation effort Sympathy and support Data format and directories =========================== The data directory contains the following three sub-directories: events/ This directory contains sub-directories for each event. In which each event directory contains tab-separated (i.e., TSV) three files, i.e., train, dev and test. Each TSV file stores ground-truth annotations for the aforementioned humanitarian categories. The data format of these files is described in detail below. event_type/ This directory contains combined event type data, we combined the training, development, and test sets of all the events that belong to the same event type. all_combined/ This directory contains the whole combined set. HumAID_ICWSM_data.jsonl: Json objects of tweets Format of the TSV files --------------------------------------------------------- Each TSV file contains the following columns, separated by a tab: tweet_id: corresponds to the actual tweet id from Twitter. tweet_text: corresponds to the tweet text. class_label: corresponds to a label assigned to a given tweet text. More details can also be found in: https://crisisnlp.qcri.org/humaid_dataset

HumAID Twitter数据集包含数千条经人工标注的推文，这些推文采集自2016至2019年间全球各地发生的19起重大自然灾害事件，涵盖地震、飓风、野火与洪水等灾害类型。该数据集是目前危机信息学领域规模最大的社交媒体数据集（约7.7万条数据），详细信息可参阅我们的研究论文。

其标注涵盖以下人道主义相关类别：
谨慎与建议
流离失所人员与疏散情况
无法判断
基础设施与公用设施损毁
伤亡人员
失踪/寻获人员
非人道主义相关
其他相关信息
请求或紧急需求
救援、志愿行动或捐赠活动
同情与支持

数据格式与目录结构
===========================
数据目录包含以下三个子目录：
1. events/：该目录为每起灾害事件分别设立子目录。每个事件子目录下均包含三个制表符分隔（TSV）格式的文件，即训练集、开发集与测试集。每个TSV文件均存储了前述人道主义类别的真实标注，下文将详细说明此类文件的数据格式。
2. event_type/：该目录存储合并后的事件类型数据，我们将属于同一事件类型的所有事件的训练集、开发集与测试集进行了合并。
3. all_combined/：该目录存储全部合并后的完整数据集。
此外还包含HumAID_ICWSM_data.jsonl：推文的JSON对象

TSV文件格式
---------------------------------------------------------
每个TSV文件包含以下制表符分隔的列：
tweet_id：对应Twitter平台上的推文唯一标识符
tweet_text：对应推文文本内容
class_label：对应为该推文文本分配的类别标签

更多详细信息可参阅：https://crisisnlp.qcri.org/humaid_dataset