Comparing Machine Learning and Heuristic Approaches for Metric-Based Code Smell Detection

Code smells represent poor implementation choices applied by developers when enhancing source code. Their negative impact on source code maintainability and comprehensibility has been widely shown in the past and several techniques to automatically detect them have been devised. Most of these techniques are based on heuristics, namely they compute a set of code metrics and combine them by creating detection rules; while they have a reasonable accuracy, a recent trend is represented by the use of machine learning where code metrics are used as predictors of the smelliness of code artifacts. Despite the recent advances in the field, there is still a noticeable lack of knowledge of whether machine learning can actually be more accurate than traditional heuristic-based approaches. To fill this gap, in this paper we propose a large-scale study to empirically compare the performance of heuristic-based and machine-learning-based techniques for metric-based code smell detection. We consider six code smell types and compare machine learning models with Decor, a state-of-the-art heuristic-based approach. Key findings emphasise the need of further research aimed at improving the effectiveness of machine learning to code smell detection. Indeed, on the one hand​, machine-learning-based-techniques generally achieve better performance without requiring any threshold definitions, while on the other hand for some specific code smell types sometimes the training data are not enough to make the models able to discriminate the smelliness of code components.

代码坏味道（Code smells）指开发者在迭代优化源代码时所采用的不良实现方案。过往研究已充分证实其对源代码可维护性与可理解性的负面影响，学界也已提出多种自动检测代码坏味道的技术手段。此类技术大多基于启发式规则：即通过计算一组代码度量指标，并结合检测规则进行组合；尽管这类方法已具备较为合理的准确率，当前学界的新兴趋势则是采用机器学习方法，以代码度量指标作为代码制品坏味道程度的预测依据。尽管该领域近年已有诸多进展，但学界仍缺乏足够的实证认知，无法明确机器学习方法是否确实优于传统启发式手段。为填补这一研究空白，本文开展了一项大规模实证研究，对比基于代码度量指标的代码坏味道检测中，启发式方法与机器学习方法的性能表现。本次研究涵盖六种代码坏味道类型，并将机器学习模型与当前主流的启发式方法Decor进行对比。核心研究结果凸显了开展进一步研究的必要性，以提升机器学习方法在代码坏味道检测中的效能。具体而言，一方面，基于机器学习的技术通常无需设定任何阈值即可获得更优性能；另一方面，针对部分特定类型的代码坏味道，训练数据往往不足，导致模型无法有效区分代码组件的坏味道程度。