Replication Package: Writing Better Software Explanations: A Guideline-Based Approach

Description This replication package accompanies the study Writing Better Software Explanations: A Guideline-Based Approach, accepted at the 2026 IEEE 34th International Requirements Engineering Conference (RE). The study investigates how high-quality software explanations can be systematically supported through a guideline-driven approach and how such a guideline can be operationalized in a Large Language Model (LLM)-based formulation assistant. The research follows a sequential mixed-method design comprising (i) elicitation of candidate explanation quality properties from literature and user interviews, (ii) prioritization through a quantitative survey, (iii) derivation of a formulation guideline, and (iv) operationalization and evaluation of the guideline in a web-based LLM-supported tool. The tool supports developers in generating, checking, and iteratively revising explanations while maintaining human control over content. The package includes: data/survey-1-property-prioritization-results.xlsx: Anonymized dataset of the guideline prioritization survey containing participant demographics and property usefulness ratings. data/developer-study-manual-vs-tool-explanations.xlsx: Developer study dataset including explanation needs, grounding content, paired manual and tool-supported explanations, formulation times, and aggregated user-study ratings. data/survey-2-user-perceived-quality-results.xlsx: Anonymized user evaluation dataset comparing perceived quality of manual and tool-supported explanations, including ratings, preferences, and qualitative feedback. analysis/: Scripts and input files for reproducing the main reported results and the diverging bar chart of property ratings. source-code/: Full implementation of the web-based guideline-driven formulation tool with prompt design, interface logic, and quality-check mechanisms. prompt-templates.md: Summary of the generation, quality-check, and quick-fix prompts used by the tool. tool-interface-screenshot.png: Screenshot of the formulation support tool. A detailed README is included with folder structure, file descriptions, installation instructions, reproduction steps, and reuse instructions. This artifact supports research on software explainability, human-AI collaboration, requirements communication, and LLM-assisted writing support. It enables replication of the guideline derivation process, reproduction of the main reported analyses, secondary analysis of explanation quality perceptions, and experimentation with guideline-driven LLM support approaches. Authors Martin Obaidi, Jean-Carl Kremser, Hannah Deters, Jakob Droste, Marc Herrmann, Kurt Schneider Citation If you use this dataset, the accompanying tool, or the study materials, please cite the accompanying paper: Obaidi, M., Kremser, J.-C., Deters, H., Droste, J., Herrmann, M., Schneider, K. (2026).Writing Better Software Explanations: A Guideline-Based Approach.In: 2026 IEEE 34th International Requirements Engineering Conference (RE). Please also cite the replication package: Obaidi, M., Kremser, J.-C., Deters, H., Droste, J., Herrmann, M., Schneider, K. (2026).Replication Package: Writing Better Software Explanations: A Guideline-Based Approach.Zenodo. https://doi.org/10.5281/zenodo.20397069. Contact Martin Obaidi ([e-mail]) License Unless otherwise noted, all data, source code, scripts, and study materials are licensed under the Creative Commons Attribution 4.0 International License (CC BY 4.0).