Detecting Code Smells using Machine Learning Techniques: Are We There Yet?

Code smells are symptoms of poor design and implementation choices weighing heavily on the quality of produced source code. During the last decades several code smell detection tools have been proposed. However, literature shows that results of these tools can be subjective and are intrinsically tied to the nature and approach of the detection. In a recent work Arcelli Fontana et al. showed how Machine-Learning (ML) techniques can be effectively adopted for detecting and acting upon code smells, possibly solving the issue of tool subjectivity giving to a learner the ability to discern between smelly and non-smelly source code elements. While this work opened a new perspective for code smell detection, in the context of our research we found a number of possible limitations that might threaten the results of the study by Arcelli Fontana et al., namely: (i) the metric distribution of smelly elements, which strongly differs from the the one of non-smelly instances; (ii) the balance between smelly and non-smelly elements; (iii) the validation methodology, and (iv) the reliance on threshold metrics used to interpret the performance of the models. In this work, we start addressing these limitations by focusing on the first issue, i.e., trying to understanding whether the results of the study by Arcelli Fontana et al. are actually due to the distribution of the metrics of smelly and non-smelly source code elements in the dataset exploited by the reference study. Our findings show that the high performance achieved in the study by Arcelli Fontana et al. was in fact due to the specific dataset employed rather than the actual capabilities of machine-learning techniques for code smell detection.