Reproduction Package for STTT Submission `Cooperative Verification: A Literature Review'

This artifact contains the aggregated data used for the article “Cooperative Verification: A Literature Review”. The artifact consists of the following data files: |– stage-1_search-space.csv<br> |– stage-2-3-4_keyword-search_process-title-abstract.csv<br> |– review_sheet.csv <strong>Search space for literature review</strong> The file <code>stage-1_search-space.csv</code> contains metadata for the articles comprising the search space for our literature review. These data correspond to the output of <code>Stage 1</code> in the methodology described in the paper. <strong>Filtering process</strong> The file <code>stage-2-3-4_keyword-search_process-title-abstract.csv</code> contains the data corresponding to the stages 2, 3, and 4 of the methodology described in the paper. It contains metadata for the articles that passed the filter of <code>keyword-search</code>, and the decisions based on reviewing titles and abstracts. <strong>Review sheet</strong> The file <code>review_sheet.csv</code> contains metadata for the articles we reviewed. It contains the information related to the application of our definition to the techniques presented in these articles, as well as the class assigned to the cooperative techniques. <strong>Generating numbers</strong> Following commands can be executed to reproduce the numbers used in our literature review. <pre><code># Change directory to the directory containing the CSV files of this artifact. # Number of papers in our search space. cat stage-1_search-space.csv | tail -n +2 | wc -l # Number of papers after keyword search. cat stage-2-3-4_keyword-search_process-title-abstract.csv | tail -n +2 | wc -l # Number of papers that are excluded based on titles. cut -f11 stage-2-3-4_keyword-search_process-title-abstract.csv | tail -n +2 | sort | uniq -c # Number of papers that are excluded based on abstracts. cut -f12 stage-2-3-4_keyword-search_process-title-abstract.csv | tail -n +2 | sort | uniq -c # Number of papers in different combination classes. cut -f8 review_sheet.csv | tail -n +2 | sort | uniq -c</code></pre>