Replication Package for "Smells Depend on the Context: An Interview Study of Issue Tracking Problems and Smells in Practice"

Purpose These artefacts accompany a research article accepted to the TOSEM journal, titled: "Smells Depend on the Context: An Interview Study of Issue Tracking Problems and Smells in Practice". With this replication package, you can investigate the original interview protocols, the results of the interview script analysis, and view a summary of the participant agreement on smell occurrence and how problematic they are. The artefacts include PDFs, Excel, and READMEs. There are no code artefacts for this research article. Utilising these artefacts requires only the Excel program, or access to Google Sheets where you can upload and access this file as a GoogleSheet. For more details regarding the scientific work, check out the title and abstract below, as well as our full article available at https://doi.org/10.48550/arXiv.2601.04124. Provenance Title: Smells Depend on the Context: An Interview Study of Issue Tracking Problems and Smells in Practice Abstract: Issue Tracking Systems (ITSs) enable software developers and managers to collect and resolve issues collaboratively. While researchers have extensively analysed ITS data to automate or assist specific activities such as issue assignments, duplicate detection, or priority prediction, developer studies on ITSs remain rare. Particularly, little is known about the challenges Software Engineering (SE) teams encounter in ITSs and when certain practices and workarounds (such as leaving issue fields like “priority” empty) are considered problematic. To fill this gap, we conducted an in-depth interview study with 26 experienced SE practitioners from different organisations and industries. We asked them about general problems encountered, as well as the relevance of 31 ITS smells (aka potentially problematic practices) discussed in the literature. By applying Thematic Analysis to the interview notes, we identified 14 common problems including issue findability, zombie issues, workflow bloat, and lack of workflow enforcement. Participants also stated that many of the ITS smells do not occur or are not problematic. Our results suggest that ITS problems and smells are highly dependent on context factors such as ITS configuration, workflow stage, and team size. We also discuss potential tooling solutions to configure, monitor, and visualise ITS smells to cope with these challenges. Authors: All authors are affiliated with the University of Hamburg, in Hamburg, Germany Lloyd Montgomery - ORCiD - [e-mail] Clara Lüders - ORCiD - [e-mail] Christian Rahe - ORCiD - [e-mail] Walid Maalej - ORCiD - [e-mail] Please cite this work as: Lloyd Montgomery, Clara Lüders, Christian Rahe, and Walid Maalej. 2026. Smells Depend on the Context: An Interview Study of Issue Tracking Problems and Smells in Practice. ACM Trans. Softw. Eng. Methodol. 1, 1 (January 2026), 30 pages. https://doi.org/10.1145/3788876 Artifact: https://doi.org/10.5281/zenodo.18170919 Data In our replication package you will find a number of individual files. Here we describe them in more detail: 1. Interview Protocol - Interview Script The full interview script we used during each interview. This was not visible to the interviewees. 1. Interview Protocol - Slides.pdf A PDF of the slides we used during each interview to guide the interview process. Theses slides closely mirror the "Interview Structure" above. These slides were "presented" to the interviewees, one slide at a time. 2. Interview Analysis Inside this excel file you will find all of our analysis data and functions for values and figures presented in the article. Here we describe the sub-sheets within the excel sheet. 💾 Script The interview script used to conduct the interviews. 💾 Interviewers The anonymised list of 5 interviewers (will be un-anonymised after article acceptance). 💾 Participants The anonymised list of 26 interviewees. 💾 Interview Data The anonymised interview data. RQ1 TA Phase 2 - {I1,I2,I5} These three sheets are the individual analyses performed by the first three authors. In each file you'll find the name of the pattern found by the researcher, and the evidence IDs that trace back to the original statements made by the participants. Note: the original statements made by the participants are not shared in accordance with the interview agreement. RQ1 TA Phase 3 This sheet contains the final state of the Thematic Analysis after Phase 3 was complete. RQ1 TA Phases 4 & 5 This sheet contains the final state of the Thematic Analysis after Phase 5 was complete. This is the complete Thematic Map. RQ2 Closed Coding This sheet contains the codes after both coders met to resolve all disagreements. Note: Not all codes for each individual statement align across the two coders. What is important, is that for each block of statements, representing one participant and one smell, that block has alignment between the two coders. For example, P01 SB2.01 (which is S1.1 in the paper), the coders agree that the P01 thinks that SB2.01 problematicness "depends" (agreement), even though the second coder has labelled two more statements as "depends" than the first coder. RQ2 Figure This is Figure 1 in the paper. It is generated automatically from the "SC - After Agreement" subsheet. The paper explains how to interpret the figure. Here is another short description: Green: Yes, they agreed the smell occurs or is problematic Red: No, they do not agree it occurs or is problematic Blue: It Depends, they believe the smell is only problematic in certain contexts Orange Exclamation (excel formula only): There is a disagreement in the coding that still needs to be resolved. These have all been resolved prior to writing the paper, but the formula still supports this if the data is changed. Nothing: They did not comment on the smell, or they commented but their comment did not fall into any of the colour codes above. RQ3 Analysis Analysis results from looking into participant responses on tooling solutions. Setup There are no system requirements beyond a PDF reader and the ability to open an Excel file (via Excel, or importing into another CSV program such as Google Sheets). Usage The PDFs provide additional information on the scientific processes conducted in the article. The Excel file contains the results of the analysis process, including the figure in the article, and the formulas used to calculate the numbers in the article. Licences Code Licence There is no code associated with this scientific article. Data and Other Non-Code Scientific Artefacts The data is licensed under CC BY 4.0. Replication Package for: "Smells Depend on the Context: An Interview Study of Issue Tracking Problems and Smells in Practice" © 2026 by Lloyd Montgomery, Clara Lüders, Christian Rahe, and Walid Maalej is licensed under CC BY 4.0. To view a copy of this license, visit https://creativecommons.org/licenses/by/4.0/