When Disagreement Becomes Territory: Plural Ground Truth in Human and Classifier Emotion Interpretation — Derived Data and Reproducible Analysis Package

This record provides a derived dataset and full reproducible analysis package for the study “When Disagreement Becomes Territory: Plural Ground Truth in Human and Classifier Emotion Interpretation.”

The package supports the analysis of how disagreement in fine-grained emotion interpretation is geometrically structured across human raters and classifier systems. It is derived from GoEmotions (Demszky et al., 2020), a publicly released fine-grained emotion annotation corpus of English-language Reddit comments annotated by multiple human raters. The present release does not redistribute the original raw text or raw GoEmotions CSV files. It contains derived feature tables, aligned probability matrices, classifier outputs, aggregate statistics, analysis scripts, and an audit log.

The dataset includes derived representations of human annotation distributions, aligned classifier outputs from SamLowe/roberta-base-go_emotions and cirimus/modernbert-large-go-emotions, and computed geometric features including entropy (H), modal dominance (M), effective-label count (D), sparsity, regime classification, Jensen-Shannon divergence, modal agreement, category baselines, and minority under-activation measures. All released tables are aligned at the instance level for N = 56,451 texts after excluding examples marked “very unclear” and retaining texts with at least three raters.

The package provides:

* Probability-simplex representations of human rater distributions across 28 emotion categories* Simplex-normalized and raw sigmoid classifier outputs for two GoEmotions-trained classifiers* Derived geometric measures and four-regime disagreement classifications* Human-model and model-model comparison metrics, including Jensen-Shannon divergence and modal agreement* Category-level baselines, active-rate measures, minority-rate measures, and dispersion statistics* Minority under-activation analyses across emotion categories and linguistic groupings* Robustness outputs under alternative normalization, thresholding, length-band, and frequency-control conditions* Executable Python scripts for preprocessing, feature construction, model inference, comparison metrics, and robustness checks* An audit log documenting file integrity, checksums, index alignment, simplex constraints, recomputation checks, and raw-text exclusion

No raw Reddit text is included. No original GoEmotions CSV files are included. Users who need full reconstruction from source should retrieve the original GoEmotions data from the Google Research repository and run the provided scripts. This design preserves reproducibility while avoiding redistribution of the source text.

This dataset does not claim to establish a normative ground truth for emotion interpretation; rather, it characterizes the empirical geometry of disagreement observed in human annotations and its systematic transformation under classifier inference.

This dataset is intended for research on affective computing, emotion AI, annotator disagreement, perspectivist NLP, human-AI disagreement, emotion representation, interpretive variability, and the epistemic limits of classifier-based emotion inference.