Replication package for EMSE article: "Retrospective Cohort Study In Action: A Pilot Investigation on the SonarQube Issue Impacting the Development Velocity"

Replication Package This replication package contains the Python, R, and notebook code, the figures, and the datasets required to conduct the data collection, preprocessing, matching, and statistical analysis of this study. Contents This repository contains the following: README.md: Main documentation for the replication package. README.html: HTML version of the repository documentation. DATA_LICENSE: License information for the dataset contents. SCRIPT_LICENSE: License information for the source code. Article_figures: Figures used in the manuscript, together with additional supporting plots generated during the analysis. CohortMethodology: Cohort methodology figures. cohort_outline_continuous.pdf: Outline of a cohort study with a continuous outcome. RetrospectiveCohort: Figures describing retrospective cohort timelines and confounding. confounding.pdf: Relationships between exposure, outcome, and confounders. thesis_timelines_fixed.pdf: Fixed follow-up date timeline. thesis_timelines_relative.pdf: Relative follow-up date timeline. StudyDesign: Study design and analysis workflow figures. SQ_cohort_design.pdf: Retrospective cohort study design. analysis-schema-short.pdf: Data analysis workflow. data-collection.pdf: Data collection process. relationships-diagram.pdf: DAG showing variable relationships. study-design.pdf: Overall study design. velocity_calculation.pdf: Example of velocity calculation. Results: Figures included in the manuscript results section. SQ_cohort_subject_selection.pdf: Subject filtering and selection workflow. desc-boxplots.pdf: Exploratory boxplots of modelled confounders. glm-residuals.pdf: Residual diagnostics for the GLM. loveplot.pdf: SMD love plot for confounder balance. sq_normality_plot.pdf: Distribution and Q-Q plots for the dependent variable by group. vif-results.pdf: VIF barplot for multicollinearity diagnostics. Matching_plots: Supporting plots for the matching stage, including histograms, boxplots, eCDF, Q-Q, density, and propensity score plots. regression_plots: Supporting regression diagnostic plots. mikelFigures: Additional exploratory figures. Additional root-level figures in Article_figures/, including: cohort_outline.pdf cohort_outline_numeric.pdf confounder_boxplot.pdf confounder_boxplots.pdf confounder_boxplots.png docker_developmentSpeed.pdf variable_boxplots.pdf codes: Source code used throughout the study. Python codes aggregateExp.py: Aggregates overall project experience from activities before and during follow-up. calculate_velocity.py: Calculates project development velocity. clone_projects.py: Clones the selected repositories. commons.py: Stores shared paths and global variables. contributors_api.py: Extracts contributor-related information from project repositories. format_data_for_analysis.py: Reformats variables for the statistical analysis stage. get_commits.py: Fetches and stores project commits when repository-level correction is needed. get_confounders_from_repos.py: Extracts confounders such as size and complexity from cloned repositories. get_developer_experience.py: Quantifies developer experience. mergeAttributes.py: Merges attributes collected across multiple stages into a unified file. repo_experience.py: Aggregates developer experience into a project-level metric. R codes confounder-matching_EMSE.R: Performs the matching stage to improve balance and overlap. crudeanalysis_EMSE.R: Performs the crude, unadjusted analysis. data-transformation_EMSE.R: Performs multicollinearity checks and related transformations. descanalysis_EMSE.R: Performs the descriptive analysis. regressionanalysis_EMSE.R: Performs the adjusted regression analysis. notebooks: Jupyter notebooks used in data collection and preprocessing. apacheGithub.ipynb: Downloads project metadata from the selected Apache projects. collectionMerge.ipynb: Merges and cleans collected attributes into the final analysis dataset. commitCrawler.ipynb: GitHub API crawler to mine project commit data. issueCrawlerGithub.ipynb: GitHub API crawler to mine issue data from GitHub repositories. jiraCrawler.ipynb: Jira API crawler to mine issue data from Jira repositories. sonarQubeCrawler.ipynb: SonarQube API crawler to identify and mine SonarQube-related project data. datasets: Data required to reproduce the study. data_analysis_R: Final dataset used for the R-based data analysis stage. SQ_cohort_data_in_days.csv: Final analysis-ready dataset. JiraDataCrawler: Jira issue mining data and supporting resources. 1.DataDownload: Download stage resources and raw Jira project issue files. DownloadData.ipynb: Notebook used to download Jira data. README.md: Documentation for the Jira download stage. jiraProjectIssues: Raw Jira issue files for Apache projects. jira_field_information.json: Jira field metadata. jira_issuelinktype_information.json: Jira issue link type metadata. jira_issuetype_information.json: Jira issue type metadata. requirements-manual.txt: Additional manual requirements for the Jira crawler stage. jira_data_sources.json: Jira data source configuration. commonIssueFolder: Unified issue tables across projects. githubIssueTables(TimeStamp): Timestamp-based GitHub issue tables. experience: Project-level developer experience data. repo_structure.txt: Text export of the repository tree. requirements.txt: Python dependencies required to run the Python scripts and notebooks. Getting Started These instructions will help you run the replication package locally. Prerequisites Running the code requires Python 3.9 or newer. Installation instructions can be found here. Install the Python dependencies with: pip install -r requirements.txt You may also want to use a virtual environment. Running the R code requires an R installation and, optionally, RStudio for a more convenient workflow. For installing the necessary R packages: install.packages("package") library(package) The required R packages depend on the specific script being executed. Running the code Important: some scripts use hard-coded paths. Update the path variables in the Python and R files as needed for your local environment. Workflow 1. Data Mining Mine project metadata from GitHub: Run notebooks/apacheGithub.ipynb Mine SonarQube/SonarCloud project information: Run notebooks/sonarQubeCrawler.ipynb Mine issue data: Run notebooks/issueCrawlerGithub.ipynb for GitHub issues Run notebooks/jiraCrawler.ipynb for Jira issues Mine commit data: Run notebooks/commitCrawler.ipynb Clone repositories for repository-based measurements: Run codes/clone_projects.py Collect developer experience: Repository-level: codes/get_commits.py codes/get_developer_experience.py Contributor/GitHub-level: codes/contributors_api.py codes/repo_experience.py Collect repository-based confounders: codes/get_confounders_from_repos.py Merge collected attributes: codes/mergeAttributes.py codes/aggregateExp.py 2. Preprocessing Merge common issues across Jira and GitHub: Run notebooks/collectionMerge.ipynb Calculate development velocity: Run codes/calculate_velocity.py Prepare the final dataset for analysis: Continue the remaining preprocessing steps in notebooks/collectionMerge.ipynb Reformat variables for the analysis stage: Run codes/format_data_for_analysis.py 3. Data Analysis Descriptive analysis: Run codes/R_codes/descanalysis_EMSE.R Crude analysis: Run codes/R_codes/crudeanalysis_EMSE.R Multicollinearity and transformation checks: Run codes/R_codes/data-transformation_EMSE.R Matching: Run codes/R_codes/confounder-matching_EMSE.R Adjusted regression analysis: Run codes/R_codes/regressionanalysis_EMSE.R Repository Structure . ├── Article_figures/ │ ├── CohortMethodology/ │ ├── Matching_plots/ │ ├── Results/ │ ├── RetrospectiveCohort/ │ ├── StudyDesign/ │ ├── mikelFigures/ │ ├── regression_plots/ │ └── ... ├── codes/ │ ├── R_codes/ │ ├── aggregateExp.py │ ├── calculate_velocity.py │ ├── clone_projects.py │ ├── commons.py │ ├── contributors_api.py │ ├── format_data_for_analysis.py │ ├── get_commits.py │ ├── get_confounders_from_repos.py │ ├── get_developer_experience.py │ ├── mergeAttributes.py │ └── repo_experience.py ├── datasets/ │ ├── JiraDataCrawler/ │ ├── commonIssueFolder/ │ ├── data_analysis_R/ │ ├── experience/ │ └── githubIssueTables(TimeStamp)/ ├── notebooks/ │ ├── apacheGithub.ipynb │ ├── collectionMerge.ipynb │ ├── commitCrawler.ipynb │ ├── issueCrawlerGithub.ipynb │ ├── jiraCrawler.ipynb │ └── sonarQubeCrawler.ipynb ├── DATA_LICENSE ├── README.html ├── README.md ├── SCRIPT_LICENSE ├── repo_structure.txt └── requirements.txt