Dataset for the Paper: "Security Defect Detection via Code Review: A Study of the OpenStack and Qt Communities"

This is the dataset for the paper: "Security Defect Detection via Code Review: A Study of the OpenStack and Qt Communities ", including the extracted data and results.

The dataset contains the following three folders:

1. RQ1: 

Security defect in Nova.xlsx

Security defect in Neutron.xlsx

Security defect in Qt Base.xlsx

Security defect in Qt Creator.xlsx;

The RQ1 folder contains four files corresponding to the four projects (i.e., Nova and Neutron from OpenStack, Qt Base and Qt Creator from Qt), including 539 security-related review comments, in which security defects were identified by the reviewers. These instances were obtained from manual labelling after keyword-based search. The security defect type of these instances are  presented to answer RQ1.

How to Read the MS Excel files in RQ1:

Each of the four MS Excel files in this folder contains 6 sheets for six years from 2017 to 2022. Each sheet has 10 columns for recoding 10 data items, among which the last four data items are used in our study to answer the RQs. We list the data items in the following table.

Data Item
Description
Source

Keyword
The corresponding keyword of the comment.
Keyword-based Search

Code_change_id
The code_change_id of the comment.
Gerrit

File
The file in which the comment is added.
Gerrit

Patchset
The patchset of the comment within the code change.
Gerrit

Line
The line number in the file at which the comment is added.
Gerrit

Message
The text of the review comment.
Gerrit

Security-related
Whether the review comment is security-related (i.e., Yes or No).
Labelling

Security defect type
The type of the security defect identified in the comment.
Labelling

Consequence
The Consequence of the security defect.
Extraction

Resolution Evidence
The information about where the identified security defect was resolved in the code
Extraction

2. RQ2: 

Extracted data for RQ2.mx22

The RQ2 folder contains the extracted data of 539 security-related review comments in Extracted data for RQ2.mx22, which was encoded and analyzed by the MAXQDA tool, investigating the treatment of security defects by developers and reviewers to answer RQ2.

3. RQ3: 

Extracted data for RQ3.mx22

The RQ3 folder contains the extracted data of 161 review comments in which identified security defects were not resolved by developers in Extracted data for RQ3.mx22. which was also encoded and analyzed by the MAXQDA tool, exploring the causes of not resolving security defects to answer RQ3.

Note: The mx22 can be opened by MAXQDA 22, which are available at https://www.maxqda.com/ for download. You may also use the free trial version of MAXQDA 2022, which is available at https://www.maxqda.com/trial for download.