DeModify

deModify consists of 3631 instances, each with three annotations obtained through CrowdFlower. An instance is a short story in which a modifier is annotated with respect to its impact on the information in the story, assessed through its deletion from the context: crucial, not-crucial, ungrammatical. Based on these annotations we have created two gold standards: strict (includes instances on which all annotators agree) and relaxed (majority voting). The archive contains a file that proposes a split into 5 folds for the instances that belong to either of the gold standards.