Artifacts for [On Rank Aggregating Test Prioritizations]
收藏资源简介：
Artifacts for [On Rank Aggregating Test Prioritizations]<br> ===================================================== <pre><code>username: entp password: entp root password: entp</code></pre> Directory structure for [/home/entp/<strong>EnTP</strong>]<br> -----------------------------------------------------------------<br> [<strong>benchmarks</strong>] -> contains the projects/subjects under test used in our study. [<strong>cost_cov_diff</strong>] -> pre-recorded cost (cachegrind's I-Ref count) per test-case. We recorded this precporcessing to avoid fluctuations in costs across different systems, and hence maintain uniformity. [<strong>raw_data_scripts</strong>] -> contains C++ implementation of EnTP, and scripts to generate results (tables, boxplots, .csv, .txt, etc.) [<strong>exp_res_raw</strong>] -> contains pre-recorded experiments results (as .csv files) as reported in the paper. To generate the plots, run [python3 plot.py] from {/home/entp/EnTP/exp_res_raw}. The plots will be generated in .eps format. These precomputed results helps the artifact reviewer save time by generating the plots only, otherwise the whole process (detailed in Makefile's usage) generates a [database] directory of ~400GB (all benchmarks, all combinations, all results even beyond those reported in the paper, + some extra logs) which is very time consuming (~1 month on a standard laptop with 8 cores, 8GB RAM). <strong>Makefile</strong>'s usage<br> ---------------------------<br> [Step 1] <pre><code class="language-bash">entp@entp:~/EnTP$ make -s entp_all_[benchmark]</code></pre> Possible values of [benchmark] = {c4, gravity, mlisp, replace, schedule2, space, xc, cf, grep, printtokens, scd, sed, tcas, xxhash, flex, gzip, printtokens2, schedule, slre, totinfo}. example: <pre><code class="language-bash">entp@entp:~/EnTP$ make -s entp_all_slre</code></pre> executes EnTP and state-of-the-arts on the benchmark "slre". Please follow log messages displayed after executing the above command. At the end of the execution, a sub-diretory named [<strong>database</strong>] will store the results for all experiments performed on "slre" for the current system and environment.<br> ...<br> [<em>database/{100, 75, 50, 25}</em>] -> contains experimental results for the consensus budget of top-{100%, 75%, 50%, 25%}.<br> ... (optional) <pre><code class="language-bash">entp@entp:~/EnTP$ make -s entp_all_[benchmark] #other benchmarks</code></pre> [Step 2] <pre><code class="language-bash">entp@entp:~/EnTP$ make -s generate_tabs</code></pre> This will collect data from the newly generated directories and results (at the end of previous step), and generate tables, and .eps plots. Check for results reproduced<br> ---------------------------------------------<br> You can visually compare the .eps plots in {<strong><em>/home/entp/EnTP</em></strong>} with the ones generated in the directory {<strong><em>/home/entp/EnTP/exp_res_raw</em></strong>}.<br> You can also compare the .csv files under these directories for quantitative comparison with some tolerance. (optional) <pre><code class="language-bash">entp@entp:~/EnTP$ make -s destroy_all</code></pre> Cleans up everything!
《排名聚合测试优先级排序》配套实验工件 ===================================================== 用户名：entp 密码：entp 管理员密码：entp /home/entp/**EnTP** 目录结构： ----------------------------------------------------------------- **基准测试集（benchmarks）**：包含本研究中使用的待测项目与测试对象。 **cost_cov_diff**：存储每条测试用例预先记录的开销（基于cachegrind的I-Ref计数）。我们预先完成该预处理步骤，以避免不同系统间的开销波动，确保实验一致性。 **raw_data_scripts**：包含EnTP的C++实现代码，以及用于生成各类结果（表格、箱线图、.csv文件、.txt文件等）的脚本。 **exp_res_raw**：存储本论文中报道的预记录实验结果（以.csv文件形式保存）。若需生成图表，请在{/home/entp/EnTP/exp_res_raw}路径下执行`python3 plot.py`命令，生成的图表将以.eps格式输出。这些预计算的结果可帮助工件评审者仅需生成图表即可验证结果，否则完整流程（详见Makefile使用说明）将生成约400GB的**database（数据库目录）**（包含所有基准测试集、所有组合方式、甚至超出论文报道范围的全部实验结果，外加额外日志文件），耗时极长（在配备8核CPU、8GB内存的标准笔记本电脑上运行约需1个月）。 **Makefile使用说明** --------------------------- [步骤1] bash entp@entp:~/EnTP$ make -s entp_all_[benchmark] [benchmark]的可选取值为：{c4、gravity、mlisp、replace、schedule2、space、xc、cf、grep、printtokens、scd、sed、tcas、xxhash、flex、gzip、printtokens2、schedule、slre、totinfo}。示例： bash entp@entp:~/EnTP$ make -s entp_all_slre 该命令将针对基准测试集“slre”运行EnTP与当前前沿测试优先级方法。请关注执行上述命令后输出的日志信息。执行结束后，名为**database**的子目录将存储当前系统与环境下针对“slre”的所有实验结果。 …… *database/{100, 75, 50, 25}*：存储针对Top-100%、Top-75%、Top-50%、Top-25%共识预算的实验结果。 …… （可选）执行： bash entp@entp:~/EnTP$ make -s entp_all_[benchmark] # 替换为其他基准测试集名称 [步骤2] bash entp@entp:~/EnTP$ make -s generate_tabs 该命令将从步骤1生成的新目录与结果中收集数据，并生成表格与.eps格式图表。 结果复现验证 --------------------------------------------- 您可以将*/home/entp/EnTP*路径下生成的.eps图表与*/home/entp/EnTP/exp_res_raw*目录中的预生成图表进行视觉对比。也可以对比两个目录下的.csv文件，在允许一定误差的范围内完成量化对比。 （可选）执行： bash entp@entp:~/EnTP$ make -s destroy_all 该命令将清理所有生成的文件与目录。
