Dataset for Evaluating Abstractive Summaries of Crisis-Related Social Media

The dataset created for evaluation of summaries generated from social media posted during five natural disasters. The dataset contains: ground truth reports created by human assessor based on ERCC Echo Flash reports (5 events); summaries generated by extractive and abstractive state-of-the-art with manual annotation of category-relevant and crisis-relevant claim. The dataset with annotated social media postings — link The full description of used summarization methods, sources of data and evaluation metrics available in the paper — link