An Approach for Traceability Recovery between Bug Reports and Test Cases

Scripts and data sets used in research for An Approach for Traceability Recovery between Bug Reports and Test Cases.

(Context) Automatic traceability recovery between software artifacts may promote early detection of issues.
Information Retrieval (IR) techniques have been proposed for the task, but they differ considerably in terms of input parameters and results. It is difficult to assess results when those techniques are applied in isolation, usually in small or medium-sized software projects. Also, an overview would be more comprehensive if a Deep Learning (DL) based technique is applied, in comparison with traditional IR techniques.
(Objective) We propose an approach to recover traceability links between bug reports and test cases, which can be instantiated with a set of IR and DL techniques.
(Method) For applying and evaluating our solution, we used historical data from the Mozilla Firefox quality assurance (QA) team, on which we assessed the following IR techniques: LSI, LDA, and BM25. We also experimented with a DL architecture called Convolutional Neural Networks (CNNs) through the use of Word Embeddings.
(Results) In this context of traceability, we noticed poor performances from three out of the four studied techniques. Only the LSI technique was effective, even standing out over the state-of-the-art BM25 technique.
(Conclusions) The obtained results suggest that the semi-automatic application of the LSI technique -- with an appropriate combination of thresholds -- is feasible for real-world software projects.