Replication Package: From Quality Properties to Practice: A Guideline and Workflow for Explainability Requirements

Description This replication package accompanies the study From Quality Properties to Practice: A Guideline and Workflow for Explainability Requirements, accepted at the 2026 IEEE 34th International Requirements Engineering Conference (RE). The study investigates how explainability requirements can be formulated more systematically through a guideline-driven workflow and how such a guideline can be operationalized in a Large Language Model (LLM)-supported formulation tool. The research follows a sequential empirical design comprising (i) elicitation of candidate requirement quality properties from literature and developer interviews, (ii) prioritization of these properties through a quantitative survey, (iii) derivation of a concise formulation guideline for explainability requirements, and (iv) operationalization and evaluation of the guideline in a web-based LLM-supported tool. The tool supports users in generating, checking, and iteratively revising explainability requirements while keeping the requirements engineer responsible for interpretation, validation, and stakeholder alignment. Persistent identifier The recommended citation uses the Zenodo Concept DOI 10.5281/zenodo.20396663. This Concept DOI remains stable across artifact updates and resolves to the latest version of the replication package. Zenodo may additionally display version-specific DOIs for individual uploaded versions. License note This artifact uses a split-license model because it contains both research materials and executable source code. Unless otherwise noted, datasets, documentation, prompt templates, screenshots, expected outputs, and study materials are licensed under CC BY 4.0. The source code in source-code/ and the executable analysis scripts in analysis/ are licensed under the MIT License. The root-level LICENSE.md file defines the applicable license for each artifact part, and source-code/LICENSE.md contains the MIT License text for the web-tool source code. Package contents README.md: Main artifact documentation with artifact summary, author information, artifact location, license information, package structure, file descriptions, system requirements, installation instructions, usage instructions, reproduction steps, expected runtime, reuse instructions, privacy notes, troubleshooting, and contact information. LICENSE.md: Split-license description for research materials, datasets, documentation, prompts, screenshots, expected outputs, source code, and executable analysis scripts. data/survey-1-property-prioritization-results.xlsx: Anonymized survey export for the property prioritization study, including participant demographics and requirements-related experience, usefulness ratings for candidate requirement properties, Top-5 and Bottom-5 rankings, template-related questions, tool-feature ratings, and optional comments. data/workshop-manual-vs-tool-requirements.xlsx: Workshop dataset including 18 app-review-derived explanation needs, requirement category labels, paired manual and tool-supported explainability requirements, participant identifiers for the study conditions, and formulation times. data/survey-2-perceived-quality-results.xlsx: Anonymized online evaluation dataset comparing perceived implementability and formulation quality of manual and tool-supported requirements, including participant demographics and experience, paired ratings, optional comments, confidence ratings, and app-review helpfulness ratings. analysis/: Analysis script, expected output, and cleaned CSV input files for reproducing the main reported quantitative results. The reproduced results include property-prioritization summaries, Top-5 and Bottom-5 property rankings, formulation-time statistics, Wilcoxon tests, a log-time tool-effect analysis, perceived-quality medians and interquartile ranges (IQRs), preference counts derived from paired ratings, and app-review helpfulness distributions. source-code/: Full implementation of the web-based guideline-driven formulation tool, including prompt design, interface logic, guideline configuration, quality-check mechanisms, quick-fix functionality, score computation, OpenAI API integration, tool-specific setup instructions, an .env.example file for API-key configuration, example-walkthrough.md for a sample input and expected interaction flow, and optional Docker-based run files. prompt-templates.md: Summary of the classification, generation, quality-check, and quick-fix prompts used by the tool. tool-interface-screenshot.png: Screenshot of the web-based formulation support tool. Reproduction and reuse The analysis results can be reproduced by running the single script analysis/reproduce_main_results.py with the dependencies listed in analysis/requirements.txt. The script output can be compared against analysis/expected-output.txt. The web tool can be run locally with Node.js/npm or, for interface inspection, through the optional Docker setup in source-code/. The web tool requires a user-provided OpenAI API key for LLM-based functions such as requirement generation, quality checking, rephrasing, custom prompts, and quick fixes. The key is configured by copying source-code/.env.example to source-code/.env and setting VITE_OPENAI_API_KEY. Without an API key, the interface still starts and can be inspected, but LLM-based functions are unavailable and the tool displays an API-key warning. Because LLM outputs are non-deterministic, the package documents expected tool behavior rather than byte-identical generated tool output. This artifact supports research on explainability requirements, requirements engineering, software explainability, human-AI collaboration, and LLM-assisted formulation support. It enables inspection and reuse of the datasets, reproduction of the main reported quantitative analyses, secondary analysis of requirement quality perceptions, and experimentation with guideline-driven LLM support approaches. Authors Martin Obaidi, Jakob Droste, Hannah Deters, Marc Herrmann, Michel Krahl, Kurt Schneider Citation If you use this replication package, the accompanying tool, the analysis scripts, the datasets, or the prompt templates, please cite the accompanying paper: Obaidi, M., Droste, J., Deters, H., Herrmann, M., Krahl, M., Schneider, K. (2026).From Quality Properties to Practice: A Guideline and Workflow for Explainability Requirements.In: 2026 IEEE 34th International Requirements Engineering Conference (RE). Please also cite this replication package: Obaidi, M., Droste, J., Deters, H., Herrmann, M., Krahl, M., Schneider, K. (2026).Replication Package: From Quality Properties to Practice: A Guideline and Workflow for Explainability Requirements.Zenodo. DOI: 10.5281/zenodo.20396663. Contact Martin Obaidi ([e-mail])