Reproduction Package for STTT Submission `Cooperative Verification: A Literature Review'

This artifact contains the aggregated data used for the article “Cooperative Verification: A Literature Review”.

The artifact consists of the following data files:

|– stage-1_search-space.csv
|– stage-2-3-4_keyword-search_process-title-abstract.csv
|– review_sheet.csv

Search space for literature review

The file stage-1_search-space.csv contains metadata for the articles comprising the search space for our literature review. These data correspond to the output of Stage 1 in the methodology described in the paper.

Filtering process

The file stage-2-3-4_keyword-search_process-title-abstract.csv contains the data corresponding to the stages 2, 3, and 4 of the methodology described in the paper. It contains metadata for the articles that passed the filter of keyword-search, and the decisions based on reviewing titles and abstracts.

Review sheet

The file review_sheet.csv contains metadata for the articles we reviewed. It contains the information related to the application of our definition to the techniques presented in these articles, as well as the class assigned to the cooperative techniques.

Generating numbers

Following commands can be executed to reproduce the numbers used in our literature review.

# Change directory to the directory containing the CSV files of this artifact.

# Number of papers in our search space.
cat stage-1_search-space.csv | tail -n +2 | wc -l

# Number of papers after keyword search.
cat stage-2-3-4_keyword-search_process-title-abstract.csv | tail -n +2 | wc -l

# Number of papers that are excluded based on titles.
cut -f11 stage-2-3-4_keyword-search_process-title-abstract.csv | tail -n +2 | sort | uniq -c

# Number of papers that are excluded based on abstracts.
cut -f12 stage-2-3-4_keyword-search_process-title-abstract.csv | tail -n +2 | sort | uniq -c

# Number of papers in different combination classes.
cut -f8 review_sheet.csv | tail -n +2 | sort | uniq -c