Static Code Analysis Alarms Filtering Reloaded: an ML Approach and its Empirical Evaluation on a New Real-World Dataset

This is a replication data package for a paper titled "Static Code Analysis Alarms Filtering Reloaded: an ML Approach and its Empirical Evaluation on a New Real-World Dataset".

Paper abstract: Even though Static Code Analysis (SCA) tools are integrated into many modern software building and testing pipelines, their practical impact is still seriously hindered by the excessive number of false positive warnings they usually produce. To cope with this problem, researchers have proposed several post-processing methods that aim to filter out false hits (or equivalently identify ``actionable'' warnings) after the SCA tool produced its results. However, we found that most of these approaches are targeted (i.e., deal with only a few SCA warning types) and evaluated on synthetic benchmarks or small-scale manually collected data sets (i.e., with typical sample sizes of several hundred).

In this paper, we present a code embedding-based approach for filtering false positive warnings produced by 160 different SonarQube rule checks, one of the most widely adopted SCA tools today. We evaluate the method on a dataset containing 224,484 real-world warning samples fixed (true positive samples) or ignored (false positive samples) by the developers, which we collected from 9,958 different open-source Java projects from GitHub using a data mining approach. It is the most extensive real-world study and public dataset we know of in this area. Our method works with an accuracy of 91% (best F1-score of 81.3% and AUC of 95.3%) for the classification of SonarQube warnings.