Comparing Machine Learning and Heuristic Approaches for Metric-Based Code Smell Detection

Code smells represent poor implementation choices applied by developers when enhancing source code. Their negative impact on source code maintainability and comprehensibility has been widely shown in the past and several techniques to automatically detect them have been devised. Most of these techniques are based on heuristics, namely they compute a set of code metrics and combine them by creating detection rules; while they have a reasonable accuracy, a recent trend is represented by the use of machine learning where code metrics are used as predictors of the smelliness of code artifacts. Despite the recent advances in the field, there is still a noticeable lack of knowledge of whether machine learning can actually be more accurate than traditional heuristic-based approaches. To fill this gap, in this paper we propose a large-scale study to empirically compare the performance of heuristic-based and machine-learning-based techniques for metric-based code smell detection. We consider six code smell types and compare machine learning models with Decor, a state-of-the-art heuristic-based approach. Key findings emphasise the need of further research aimed at improving the effectiveness of machine learning to code smell detection. Indeed, on the one hand​, machine-learning-based-techniques generally achieve better performance without requiring any threshold definitions, while on the other hand for some specific code smell types sometimes the training data are not enough to make the models able to discriminate the smelliness of code components.