Manual Topic Annotation of German Novels and Parlament Protocols by multiple Annotators

This dataset was created in the research project hermA and contains topic annotations for 960 sentences, half of which were taken from transcripts of the German Bundestag and half from recent German novels.

For each sentence, 30 different annotators evaluated whether illness is addressed, how central the topic is, if so, and how certain they are in the annotation.

The dataset contains the following columns:

item_id (for each sentence)

worker_id (for each annotator)

worker_group (either "crowdworker" or "student")

corpus (either "protocol_corpus"=transcript corpus or "novel_corpus"=fiction corpus)

text_source

previous_sentences (in the text_source)

target_sentence

following_sentences (in the text_source)

semantic_field_token (if existing)

semantic_field_status (either "True" or "False")

1_wird_im_fett_gedruckten_satz_krankheit_thematisiert (topic annotations: either "ja" or "nein")

1b_wie_zentral_ist_das_thema_krankheit_im_fettgedruckten_satz (topic centrality: "NaN", "krankheit_kommt_eher_am_rande_des_satzes_vor" or "krankheit_ist_das_zentrale_thema_des_satzes")

2_wie_sicher_bist_du_dir_bei_der_antwort_zu_frage_1_ (annotation certainty: "sehr_sicher", "eher_sicher", "eher_unsicher" or "sehr_unsicher")

We use the annotations to model ambiguity in:

Andresen, Melanie; Vauth, Michael & Zinsmeister, Heike. 2020. Modeling Ambiguity with Many Annotators and Self-Assessments of Annotator Certainty. Proceedings of 14th Linguistic Annotation Workshop.