Bad Practices in Performance Testing: Insights from a Multi-Perspective Empirical Study

This repository contains the replication package for the paper “Bad Practices in Performance Testing: Insights from a Multi-Perspective Empirical Study”, a collaboration between the University of Naples Federico II, the Gran Sasso Science Institute (GSSI), and the University of Salerno. Overview Despite the crucial role of performance testing in ensuring system reliability and a good user experience, this practice is often neglected or adopted too late in the development process. One contributing factor is the lack of clear guidelines and recommendations for testers. While the concept of bad practices (often referred to as test smells) has been widely investigated in various testing activities—such as unit testing and GUI-level testing, it remains largely unexplored in the context of performance testing. This study aims to: Define a taxonomy of bad practices in performance testing. Assess practitioners’ perspectives and experiences regarding these bad practices. Evaluate their prevalence in open-source performance test artifacts drawn from the E2EGit dataset. Research Questions The study is structured around three main research questions: RQ1 — Which performance testing bad practices are most commonly identified in practitioner-oriented gray literature? RQ2 — How do practitioners perceive and experience these bad practices? RQ3 — To what extent do bad practices occur in open-source performance testing artifacts? Methodology From a methodological standpoint, the study adopts a four-phase mixed-method design, integrating qualitative and quantitative analyses to triangulate evidence from multiple sources: Gray Literature Review, extraction and systematization of bad practices discussed in practitioner-oriented sources (addresses RQ1). Practitioner Survey and Expert Interviews, validation and contextualization of the relevance of identified bad practices (addresses RQ2). Automated Detection Tool, implementation and application of a detection tool on the E2EGit dataset to assess the prevalence of selected bad practices in real-world JMeter tests (addresses RQ3). Repository Structure The replication package is organized into three main folders, each corresponding to one research question. RQ1 contains: i) A .docx file representing the final taxonomy of performance testing bad practices. For each bad practice, a table reports all the references associated with it. A .xlsx file listing all URLs reviewed during the gray literature phase by the two authors. URLs highlighted in red were deemed not relevant by both authors. RQ2 contains: A .xlsx file aggregating responses collected through the practitioner survey. A file ExpertX-fully interview.docx of interview X with the corresponding expert (video recordings are not provided for privacy reasons). A file Performance_Bad_Practices_ExpertX_Summary.docx containing a summary of the discussion for interview X, reporting insights and opinions for each bad practice. RQ3 contains: A folder containing the tool developed to automatically detect a subset of bad practices in JMeter performance tests. A folder containing, for each JMeter test included in the E2EGit dataset, an analysis of its version history and the detected bad practices. Acknowledgments We thank Mr. Vincenzo De Rosa for his contribution to the development of the automated detection tool during his bachelor's degree.