Deep Learning Benchmarks and Datasets for Social Media Image Classification for Disaster Response

The crisis image benchmark dataset consists data from several data sources such as CrisisMMD, data from AIDR and Damage Multimodal Dataset (DMD). The purpose of this work was to develop a consolidated dataset, create non-overlapping train/dev/test set and provide a benchmark results for the community. We propose new datasets for disaster type detection, and informativeness classification, and damage severity assessment. Moreover, we relabel existing publicly available datasets for new tasks. We identify exact- and near-duplicates to form non-overlapping data splits, and finally consolidate them to create larger datasets. In our extensive experiments, we benchmark several state-of-the-art deep learning models and achieve promising results. We release our datasets and models publicly, aiming to provide proper baselines as well as to spur further research in the crisis informatics community. https://crisisnlp.qcri.org/crisis-image-datasets-asonam20 The labels in the dataset for different tasks are as follows: Task 1: Disaster types Earthquake Fire Flood Hurricane Landslide Not disaster Other disaster Task 2: Informativeness Informative Not informative Task 3: Humanitarian categories Affected, injured, or dead people Infrastructure and utility damage Not humanitarian Rescue volunteering or donation effort Task 4: Damage severity Little or none Mild Severe Please cite the following papers, if you use any of these resources in your research. Firoj Alam, Ferda Ofli, Muhammad Imran, Tanvirul Alam, Umair Qazi, Deep Learning Benchmarks and Datasets for Social Media Image Classification for Disaster Response, In 2020 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM), 2020. [Bibtex] Firoj Alam, Ferda Ofli, and Muhammad Imran, CrisisMMD: Multimodal Twitter Datasets from Natural Disasters. In Proceedings of the 12th International AAAI Conference on Web and Social Media (ICWSM), 2018, Stanford, California, USA. [Bibtex] Hussein Mozannar, Yara Rizk, and Mariette Awad, Damage Identification in Social Media Posts using Multimodal Deep Learning, In Proc. of ISCRAM, May 2018, pp. 529–543.