Dataset: A German Gold-Standard Dataset for Sentiment Analysis in Software Engineering

A German Gold-Standard Dataset for Sentiment Analysis in Software Engineering

Description:This repository provides a high-quality, German gold-standard dataset for sentiment analysis in the context of German software engineering discussions. The dataset consists of 5,949 developer statements sourced from the Android-Hilfe.de forum. Each statement was annotated by three independent raters according to basic emotions (following Shaver et al.) and mapped to sentiment polarity (negative, neutral, positive) via majority voting.

The dataset was specifically developed to address the lack of German, domain-specific resources for sentiment analysis in software engineering. Along with the annotated gold-standard data, we provide comprehensive documentation of the data creation process, automated pre-filtering steps (using GerVADER), and all artifacts needed for transparency and reproducibility.

Additionally, the resource contains detailed evaluation results of four German sentiment analysis tools (GerVADER, SentiStrength_DE, TextBlobDE, and BertDE), including per-sample predictions and interrater agreement metrics. All code, intermediate files, and analysis outputs are included for replicability.

This dataset enables benchmarking, model development, and meta-analysis of sentiment analysis methods tailored to German developer discourse.

Citation

If you use this dataset, please cite the following publication:

Obaidi, M., Herrmann, M., Schmid, E., Ochsner, R., Schneider, K., & Klünder, J. (2025). A German gold-standard dataset for sentiment analysis in software engineering. In 2025 IEEE 33rd International Requirements Engineering Workshop (REW).

License

This dataset is provided under the Creative Commons Attribution 4.0 International License (CC BY 4.0).

Contact

For questions regarding the dataset, please contact the corresponding author as listed in the publication.