Replication Package: From Quality Properties to Practice: A Guideline and Workflow for Explainability Requirements

Description This replication package accompanies the study From Quality Properties to Practice: A Guideline and Workflow for Explainability Requirements, accepted at the 2026 IEEE 34th International Requirements Engineering Conference (RE). The study investigates how explainability requirements can be formulated more systematically through a guideline-driven workflow and how such a guideline can be operationalized in a Large Language Model (LLM)-supported formulation tool. The research follows a sequential empirical design comprising (i) elicitation of candidate requirement quality properties from literature and developer interviews, (ii) prioritization of these properties through a quantitative survey, (iii) derivation of a concise formulation guideline for explainability requirements, and (iv) operationalization and evaluation of the guideline in a web-based LLM-supported tool. The tool supports users in generating, checking, and iteratively revising explainability requirements while keeping the requirements engineer responsible for interpretation, validation, and stakeholder alignment. The package includes: data/survey-1-property-prioritization-results.xlsx: Anonymized survey export for the property prioritization study, including participant demographics and requirements-related experience, usefulness ratings for candidate requirement properties, Top-5 and Bottom-5 rankings, template-related questions, tool-feature ratings, and optional comments. data/workshop-manual-vs-tool-requirements.xlsx: Workshop dataset including 18 app-review-derived explanation needs, requirement category labels, paired manual and tool-supported explainability requirements, participant identifiers for the study conditions, and formulation times. data/survey-2-perceived-quality-results.xlsx: Anonymized online evaluation dataset comparing perceived implementability and formulation quality of manual and tool-supported requirements, including participant demographics and experience, paired ratings, optional comments, confidence ratings, and app-review helpfulness ratings. analysis/: Analysis script, expected output, and cleaned CSV input files for reproducing the main reported quantitative results. The reproduced results include property-prioritization summaries, Top-5 and Bottom-5 property rankings, formulation-time statistics, Wilcoxon tests, a log-time tool-effect analysis, perceived-quality medians and interquartile ranges (IQRs), preference counts derived from paired ratings, and app-review helpfulness distributions. source-code/: Full implementation of the web-based guideline-driven formulation tool, including prompt design, interface logic, guideline configuration, quality-check mechanisms, quick-fix functionality, score computation, and OpenAI API integration. The source-code README documents setup, API-key configuration, and the tool behavior when no API key is configured. prompt-templates.md: Summary of the classification, generation, quality-check, and quick-fix prompts used by the tool. tool-interface-screenshot.png: Screenshot of the web-based formulation support tool. A detailed README is included with folder structure, file descriptions, system requirements, installation instructions, usage instructions, reproduction steps, expected runtime, reuse instructions, privacy notes, and contact information. This artifact supports research on explainability requirements, requirements engineering, software explainability, human-AI collaboration, and LLM-assisted formulation support. It enables inspection and reuse of the datasets, reproduction of the main reported quantitative analyses, secondary analysis of requirement quality perceptions, and experimentation with guideline-driven LLM support approaches. Authors Martin Obaidi, Jakob Droste, Hannah Deters, Marc Herrmann, Michel Krahl, Kurt Schneider Citation If you use this replication package, the accompanying tool, the analysis scripts, or the prompt templates, please cite the accompanying paper: Obaidi, M., Droste, J., Deters, H., Herrmann, M., Krahl, M., Schneider, K. (2026).From Quality Properties to Practice: A Guideline and Workflow for Explainability Requirements.In: 2026 IEEE 34th International Requirements Engineering Conference (RE). Please also cite this replication package: Obaidi, M., Droste, J., Deters, H., Herrmann, M., Krahl, M., Schneider, K. (2026).Replication Package: From Quality Properties to Practice: A Guideline and Workflow for Explainability Requirements.Zenodo. DOI: 10.5281/zenodo.20396663. Contact Martin Obaidi ([e-mail]) License Unless otherwise noted, datasets, documentation, prompt templates, screenshots, expected outputs, and study materials are licensed under CC BY 4.0. The source code in source-code/ and the executable analysis scripts in analysis/ are licensed under the MIT License.