Replication package: Reassessing prompts and feedback after model replacement: A study of LLM-assisted requirements refinement
收藏资源简介:
Replication package for a double-anonymous conference submission. Contains the pipeline, the raw experimental outputs, and the analysis that reproduces every number and figure in the paper. The study varies two things independently in requirement refinement, the design of the refinement prompt and whether the model receives evaluative feedback on the original requirement before rewriting it, and asks whether either advantage survives a change of the model doing the rewriting. Ten prompts are run with and without feedback across four models spanning an older and a newer generation in each of two vendor families, on a development set of 200 requirements, and predictions fixed there are then tested once on a held-out 674. Quality is measured by an evaluator selected from sixteen candidates by agreement with a reference derived from five industrial practitioners, and the main development analyses are repeated with a second evaluator. No API credentials are needed to check the reported numbers. python src/analysis/run_all_analysis.py recomputes every statistic, generated table and figure after verifying each raw input artefact against a SHA-256 manifest and checking that no cell is short. See README.md for what to run, and DATA.md for which results trees back which claims.



