The Tell-Tale Hat: Surfacing the Uncertainty in Folklore Classification

This dataset contains the texts of 31,088 Danish legends and descriptions of everyday life collected by the folklorist Evald Tang Kristensen from 1867 to 1924. The texts were digitized via OCR of printed editions. We have applied basic orthographic normalization and OCR correction to the texts, but otherwise they appear as originally printed. The input.txt file contains one story per line, prefaced by two tab-delimited metadata fields: a unique identifier for the story with the following format: COLLECTION_VOLUME_CHAPTER_NUMBER and the high-level classification (of 36 possible categories) assigned to the story by the folklorist when preparing the stories for publication. The accompanying code can be run on this input or any other set of texts with a single classifier assigned to each story to generate the naïve Bayes confusion matrix described in the paper.