JIT Requirements Quality Framework

Until now quality assessment of requirements has focused on traditional up-front requirements. Just-in-time (JIT) requirements are by definition incomplete, not specific and might be ambiguous when initially specified, indicating a different notion of ‘correctness’. We analyze how assessment of JIT requirements quality should be performed, based on literature of traditional and JIT requirements. We have designed a quality framework for JIT requirements and instantiated it for feature requests in open source projects. We have translated the framework into a checklist with a scoring algorithm to evaluate feature requests. We have performed an initial qualitative validation of our checklist for feature requests with eight practitioners from the Dutch agile community, receiving overall positive feedback. Then we conducted an experiment with over 80 software engineers to apply the checklist on three open source projects. This resulted in quantitative data from over 550 feature requests, together with some qualitative feedback. We use the same data to present a summarized view of the feature request quality for the three open source projects. The data sets, check list, questionnaire and interview questions (in Dutch) can be found on-line for reference.