Replication Package: Writing Better Software Explanations: A Guideline-Based Approach

Description This replication package accompanies the study Writing Better Software Explanations: A Guideline-Based Approach, accepted at the 2026 IEEE 34th International Requirements Engineering Conference (RE). The study investigates how high-quality software explanations can be systematically supported through a guideline-driven approach and how such a guideline can be operationalized in a Large Language Model (LLM)-based formulation assistant. The research follows a sequential mixed-method design comprising (i) elicitation of candidate explanation quality properties from literature and user interviews, (ii) prioritization through a quantitative survey, (iii) derivation of a formulation guideline, and (iv) operationalization and evaluation of the guideline in a web-based LLM-supported tool. The tool supports developers in generating, checking, and iteratively revising explanations while maintaining human control over content. The package includes: data/survey-1-property-prioritization-results.xlsx: Anonymized dataset of the guideline prioritization survey containing participant demographics and property usefulness ratings. data/developer-study-manual-vs-tool-explanations.xlsx: Developer study dataset including explanation needs, grounding content, paired manual and tool-supported explanations, formulation times, and aggregated user-study ratings. data/survey-2-user-perceived-quality-results.xlsx: Anonymized user evaluation dataset comparing perceived quality of manual and tool-supported explanations, including ratings, preferences, and qualitative feedback. analysis/: Scripts and input files for reproducing the main reported results and the diverging bar chart of property ratings. source-code/: Full implementation of the web-based guideline-driven formulation tool with prompt design, interface logic, and quality-check mechanisms. prompt-templates.md: Summary of the generation, quality-check, and quick-fix prompts used by the tool. tool-interface-screenshot.png: Screenshot of the formulation support tool. A detailed README is included with folder structure, file descriptions, installation instructions, reproduction steps, and reuse instructions. This artifact supports research on software explainability, human-AI collaboration, requirements communication, and LLM-assisted writing support. It enables replication of the guideline derivation process, reproduction of the main reported analyses, secondary analysis of explanation quality perceptions, and experimentation with guideline-driven LLM support approaches. Authors Martin Obaidi, Jean-Carl Kremser, Hannah Deters, Jakob Droste, Marc Herrmann, Kurt Schneider Citation If you use this dataset, the accompanying tool, or the study materials, please cite: Obaidi, M., Kremser, J.-C., Deters, H., Droste, J., Herrmann, M., Schneider, K. (2026).Writing Better Software Explanations: A Guideline-Based Approach.In: 2026 IEEE 34th International Requirements Engineering Conference (RE). Contact Martin Obaidi ([e-mail]) License Unless otherwise noted, all data, source code, scripts, and study materials are licensed under the Creative Commons Attribution 4.0 International License (CC BY 4.0).本复现包配套于论文《撰写更优质的软件说明文档：一种基于指南的方法》（"Writing Better Software Explanations: A Guideline-Based Approach"），该论文已被2026年IEEE第34届国际需求工程会议（International Requirements Engineering Conference, RE）收录。本研究探讨了如何通过指南驱动的方法系统性支撑高质量软件说明文档的创作，以及如何将此类指南在基于大语言模型（Large Language Model, LLM）的文本生成助手落地应用。 本研究采用序贯混合研究设计，具体包括：(i) 从现有文献与用户访谈中提炼候选说明文档质量属性；(ii) 通过定量调研对属性进行优先级排序；(iii) 推导得到文本生成指南；(iv) 在基于网页的LLM辅助工具中实现该指南并开展评估。该工具支持开发者在保持人工对内容控制权的前提下，生成、核查并迭代修订说明文档。 本复现包包含以下内容： 1. data/survey-1-property-prioritization-results.xlsx：指南优先级调研的匿名数据集，包含参与者人口统计学特征与属性有用性评分。 2. data/developer-study-manual-vs-tool-explanations.xlsx：开发者研究数据集，涵盖说明文档需求、锚定内容、配对的人工与工具辅助生成的说明文档、生成时长以及汇总的用户研究评分。 3. data/survey-2-user-perceived-quality-results.xlsx：用户评估匿名数据集，用于对比人工与工具辅助生成的说明文档的感知质量，包含评分、偏好与定性反馈。 4. analysis/：用于复现主要报告结果与属性评分分歧柱状图的脚本及输入文件。 5. source-code/：基于网页的指南驱动型生成工具的完整实现代码，包含提示词设计、界面逻辑与质量核查机制。 6. prompt-templates.md：工具所使用的生成、质量核查与快速修正提示词汇总。 7. tool-interface-screenshot.png：生成辅助工具的界面截图。 包中附带详细的README文档，涵盖文件夹结构、文件说明、安装指南、复现步骤与复用说明。 本研究成果可支撑软件可解释性、人机AI协作、需求沟通以及LLM辅助写作支持领域的相关研究。其支持复现指南推导流程、复现主要报告分析、对说明文档质量感知开展二次分析，以及对指南驱动的LLM支持方法进行实验验证。 作者 Martin Obaidi, Jean-Carl Kremser, Hannah Deters, Jakob Droste, Marc Herrmann, Kurt Schneider 引用说明 若您使用本数据集、配套工具或本研究相关材料，请引用如下文献： Obaidi, M., Kremser, J.-C., Deters, H., Droste, J., Herrmann, M., Schneider, K. (2026). "Writing Better Software Explanations: A Guideline-Based Approach". In: 2026 IEEE 34th International Requirements Engineering Conference (RE). 联系方式 Martin Obaidi ([e-mail]) 许可证 除非另有说明，本数据集、源代码、脚本及所有研究材料均采用知识共享署名4.0国际许可协议（Creative Commons Attribution 4.0 International License, CC BY 4.0）进行授权。