On the test smells detection: an empirical study on the JNose Test accuracy

Data collected in the study "On the test smells detection: an empirical study on the JNose Test accuracy"

Abstract: Several strategies have been proposed for test quality measurement and analysis. Code coverage is likely the most widely used one. It enables to verify the ability of a test case to cover as many source code branches as possible. Although code coverage has been widely used, novel strategies have been recently employed. Test smell analysis, for example, has been introduced as an affordable strategy to evaluate the test code quality. Test smells are poor design choices in implementation, and their occurrence in test code might reduce the test suites quality. The test smells identification is most dependent on tool support; otherwise, it could become a cost­ineffective strategy. In an earlier study, we proposed the JNose Test, a tool to analyze test suite quality from the test smells perspec­tive. The JNose Test detects twenty­one types of test smells throughout software versions. This study extends the previous one in two directions: i) the test smells detection rules were extracted to an API, named JNose-Core, that provides an extensible architecture for the implementation of new detection rules or programming languages; and ii) we performed an empirical study to evaluate the tool effectiveness for the test smells detection and a comparison between the JNose Test and the state-­of-­the­art tool, the tsDetect. Results showed that the JNose-Core precision score ranges from 91% to 100%, and the recall score from 89% to 100%. It also presented a slight improvement in the test smells detection rules compared to the tsDetect for the test smells detection at the class ­level.