Supporting Error Chains in Static Analysis for Precise Evaluation Results and Enhanced Usability

Abstract<b>Context:</b> Static analyses are well-established to aid in understanding bugs or vulnerabilities during the development process or in large-scale studies. A low false positive rate is essential for the adaption in practice and for precise results of empirical studies. Unfortunately, static analyses tend to report where a vulnerability manifests rather than the fix location. This can cause presumed false positives or imprecise results.<b> </b><b>Method:</b> To address this problem, we designed an adaption of an existing static analysis algorithm that can distinguish between a manifestation and fix location and reports error chains. Each error chain presents the dependency between the fix location with at least one manifestation location. We used our tool for a case study of 471 GitHub repositories and conducted an expert interview to investigate usability implications of the change. Further, we benchmarked both analysis versions to compare the runtime impact.<b> Result:</b> We found that 50% of the projects with a report had at least one error chain. During our expert interview, all participants required fewer executions of the static analysis if they used our adapted version. Our performance benchmark demonstrated that our improvement caused only a minimal runtime overhead of less than 4%.<b> </b><b>Conclusion:</b> Our results indicate that error chains occur frequently in real-world projects and ignoring them can lead to imprecise evaluation results. The performance benchmark indicates that our tool is a feasible and efficient solution for detecting error-chains in real-world projects. Further, our results indicate that the usability of static analyses benefits from supporting error chains.<br>DataThis artefact as of now contains information about the expert interview, see <code>expertinterview.md</code> for more details. Further, we include the graphics for the runtime evaluation. <br>