Replication Package: Writing Better Software Explanations: A Guideline-Based Approach

Description This replication package accompanies the study Writing Better Software Explanations: A Guideline-Based Approach, accepted at the 2026 IEEE 34th International Requirements Engineering Conference (RE). The study investigates how high-quality software explanations can be systematically supported through a guideline-driven approach and how such a guideline can be operationalized in a Large Language Model (LLM)-based formulation assistant. The research follows a sequential mixed-method design comprising (i) elicitation of candidate explanation quality properties from literature and user interviews, (ii) prioritization through a quantitative survey, (iii) derivation of a formulation guideline, and (iv) operationalization and evaluation of the guideline in a web-based LLM-supported tool. The tool supports developers in generating, checking, and iteratively revising explanations while maintaining human control over content. The package includes: data/survey-1-property-prioritization-results.xlsx: Anonymized dataset of the guideline prioritization survey containing participant demographics and property usefulness ratings. data/developer-study-manual-vs-tool-explanations.xlsx: Developer study dataset including explanation needs, grounding content, paired manual and tool-supported explanations, formulation times, and aggregated user-study ratings. data/survey-2-user-perceived-quality-results.xlsx: Anonymized user evaluation dataset comparing perceived quality of manual and tool-supported explanations, including ratings, preferences, and qualitative feedback. analysis/: Scripts and input files for reproducing the main reported quantitative results of the study. source-code/: Full implementation of the web-based guideline-driven formulation tool with prompt design, interface logic, and quality-check mechanisms. prompt-templates.md: Summary of the generation, quality-check, and quick-fix prompts used by the tool. tool-interface-screenshot.png: Screenshot of the formulation support tool. A detailed README is included with folder structure, file descriptions, installation instructions, reproduction steps, and reuse instructions. This artifact supports research on software explainability, human-AI collaboration, requirements communication, and LLM-assisted writing support. It enables inspection and reuse of the datasets, reproduction of the main reported quantitative analyses, secondary analysis of explanation quality perceptions, and experimentation with guideline-driven LLM support approaches. Authors Martin Obaidi, Jean-Carl Kremser, Hannah Deters, Jakob Droste, Marc Herrmann, Kurt Schneider Citation If you use this dataset, the accompanying tool, the analysis scripts, or the prompt templates, please cite the accompanying paper: Obaidi, M., Kremser, J.-C., Deters, H., Droste, J., Herrmann, M., Schneider, K. (2026).Writing Better Software Explanations: A Guideline-Based Approach.In: 2026 IEEE 34th International Requirements Engineering Conference (RE). Please also cite the replication package: Obaidi, M., Kremser, J.-C., Deters, H., Droste, J., Herrmann, M., Schneider, K. (2026).Replication Package: Writing Better Software Explanations: A Guideline-Based Approach.Zenodo. https://doi.org/10.5281/zenodo.20399316. Contact Martin Obaidi ([e-mail]) License Unless otherwise noted, all data, source code, scripts, prompt templates, and documentation are licensed under the Creative Commons Attribution 4.0 International License (CC BY 4.0).