How are test smells treated in the wild? A tale of two empirical studies

Data collected in the study "How are test smells treated in the wild? A tale of two empirical studies".

Abstract: Developing test code may be a time­-consuming process that usually requires much effort and cost, es­pecially when done manually. Besides, during this process, developers and testers are likely to adopt bad design choices, which may lead to the introduction of the so­called test smells in the test code. As the test code with test smells increases in size, these tests might become more complex, and as a consequence, much more challenging to understand and evolve them correctly. Therefore, test smells may have a negative impact on the test code quality and maintenance and may also harm the whole software testing activities. In this context, this study aims to under­stand whether software testing practitioners unintentionally insert test smells when they implement test code. We first carried out an expert survey to analyze the usage frequency of a set of test smells and then interviews to reach a deeper understanding of how practitioners deal with test smells. Sixty professionals participated in the survey, and fifty professionals participated in the interview. The yielded results indicate that experienced professionals in­troduce test smells during their daily programming tasks, even when using their companies’ standardized practices. Additionally, test development and quality improvement are usually supported by tools, but most interviewees are test smells unaware.