Revisiting Machine Learning based Test Case Prioritization for Continuous Integration

This repository contains a replication package for a research paper submitted to the 45th International Conference on Software Engineering (https://conf.researchr.org/home/icse-2023). We provide our code, data, and result for the ease of replicating our experiments. <strong>Code</strong> In the <strong>collect_data</strong> subdirectory, scripts for constructing TCP datasets are provided. We do dependency analysis using Understand (https://www.scitools.com/), so please download the related tools in advance. In the<strong> rl</strong> subdirectory, we provide the python implementation for algorithms RL, COLEMAN, PPO2-PO, ACER-PA, PPO1-LI. In the <strong>supervised_learning</strong> subdirectory, we provide implementations for MART, RankNet, RankBoost, CA, L-MART, which mainly rely on Ranklib (https://sourceforge.net/p/lemur/wiki/RankLib/.). We also provide implementation for DeepOrder. <strong>Data</strong> The <strong>origin</strong> subdirectory contains the original datasets collected from github using our scripts, including 11 projects. The <strong>smote</strong> subdirectory contains the datasets pre-processed by SMOTE. <strong>Result</strong> Results for <strong>RQ1</strong>, <strong>RQ2</strong>, <strong>RQ3</strong>, and <strong>threats to validity</strong> are provided in the corresponding subdirectories. Scripts for plotting figures are also provided. <strong>Reference</strong> We adopt code from previous work Learning-to-Rank vs Ranking-to-Learn: Strategies for Regression Testing in Continuous Integration (https://dl.acm.org/doi/abs/10.1145/3377811.3380369) Github repository: https://github.com/icse20/RT-CI Reinforcement Learning for Test Case Prioritization (https://ieeexplore.ieee.org/abstract/document/9394799) Github repository: https://github.com/moji1/tp_rl DeepOrder: Deep Learning for Test Case Prioritization in Continuous Integration Testing (https://ieeexplore.ieee.org/abstract/document/9609187) Github repository: https://github.com/AizazSharif/DeepOrder-ICSME21 A Multi-Armed Bandit Approach for Test Case Prioritization in Continuous Integration Environments (https://ieeexplore.ieee.org/abstract/document/9086053) Github repository: https://github.com/jacksonpradolima/coleman4hcs