Reproduction Package for Submission `A Literature Review on Verification and Abstraction of Neural Networks'

This artifact contains the aggregated data for the article "A Literature Review on Verification and Abstraction of Neural Networks".

The artifact contains the following data files:

|-- phase-1_search-space.csv

|-- phase-2-3_process-title-abstract.csv

|-- review_sheet.csv

Search space for literature review

The file phase-1_search-space.csv contains the metadata of the papers published in our considered conferences during the observed years.

Filtering process

In the file phase-2-3_process-title-abstract.csv, we include the files that contain our expected keywords.

Depending on whether we decided to include the paper in our survey after reading the title and the abstract, we marked each of the papers with `accept` or `decline`.

Review sheet

The file review_sheet.csv contains metadata describing our reviews for each of the publications.

It holds information on our final review of each paper, including the group we assigned it to and the reasons for its exclusion.

Generating numbers

The following commands can be executed to reproduce the numbers used in our literature review.

# Change directory to the directory containing the CSV files of this artifact.

# Number of papers in our search space.

cat phase-1_search-space.csv | tail -n +2 | wc -l

# Number of papers after keyword search.

cat phase-2-3_process-title-abstract.csv | tail -n +2 | wc -l

# Number of papers that are excluded based on abstracts and titles.

cut -f9 phase-2-3_process-title-abstract.csv | tail -n +2 | sort | uniq -c

# Number of papers in different classes.

cut -f7 review_sheet.csv | tail -n +2 | sort | uniq -c