SemEval-2020 Task 7: Assessing Humor in Edited News Headlines

This is the task dataset for SemEval-2020 Task 7: Assessing Humor in Edited News Headlines.

The task’s dataset contains news headlines in which short edits were applied to make them funny, and the funniness of these edited headlines was rated using crowdsourcing. This task includes two subtasks, the first of which is to estimate the funniness of headlines on a humor scale in the interval 0-3. The second subtask is to predict, for a pair of edited versions of the same original headline, which is the funnier version.

CodaLab page hosting the competition:
https://competitions.codalab.org/competitions/20970

Starter Github code (scripts for running baseline and evaluation):
https://github.com/n-hossain/semeval-2020-task-7-humicroedit

Task mailing list:
https://groups.google.com/forum/#!forum/semeval-2020-task-7-all
----------------------------------------------------------------------

ZIP contents:
-------------

Folders:
    - subtask-1: Dataset for the funniness regression subtask.
    - subtask-2: Dataset for the "Funnier of the Two" classification subtask.

Files:
    - {train, dev, test}.csv: the task's dataset including labels
    - train_funlines.csv: additional training data gathered from the FunLines competition (https://funlines.co)
    - baseline.zip: contains csv file which is the output of the BASELINE system. This is a template of the output format that can be submitted to CodaLab for scoring.

Reference

Please cite the task paper when using this dataset:

Nabil Hossain, John Krumm, Michael Gamon and Henry Kautz. 2020. Semeval-2020 Task 7: Assessing Humor in Edited News Headlines. In Proceedings of International Workshop on Semantic Evaluation (SemEval-2020).

BIBTEX: 
@InProceedings{hossainSemEval2020Task7, author = {Hossain, Nabil and Krumm, John and Gamon, Michael and Kautz,Henry}, title = {SemEval-2020 {T}ask 7: {A}ssessing Humor in Edited News Headlines}, booktitle = {Proceedings of the 14th International Workshop on Semantic Evaluation ({S}em{E}val-2020)}, address = {Barcelona, Spain}, year = {2020}}