Dataset for the Paper: "Security Defect Detection via Code Review: A Study of the OpenStack and Qt Communities"

This is the dataset for the paper: "Security Defect Detection via Code Review: A Study of the OpenStack and Qt Communities ", including the extracted data and results. The dataset contains the following three folders: <strong>1. RQ1</strong>: <strong>Security defect in Nova.xlsx</strong> <strong>Security defect in Neutron.xlsx</strong> <strong>Security defect in Qt Base.xlsx</strong> <strong>Security defect in Qt Creator.xlsx;</strong> The RQ1 folder contains four files corresponding to the four projects (i.e., Nova and Neutron from OpenStack, Qt Base and Qt Creator from Qt), including 539 security-related review comments, in which security defects were identified by the reviewers. These instances were obtained from manual labelling after keyword-based search. The security defect type of these instances are presented to answer RQ1. <strong>How to Read the MS Excel files in RQ1:</strong> Each of the four MS Excel files in this folder contains 6 sheets for six years from 2017 to 2022. Each sheet has 10 columns for recoding 10 data items, among which the last four data items are used in our study to answer the RQs. We list the data items in the following table. <strong>Data Item</strong> <strong>Description</strong> <strong>Source</strong> Keyword The corresponding keyword of the comment. Keyword-based Search Code_change_id The code_change_id of the comment. Gerrit File The file in which the comment is added. Gerrit Patchset The patchset of the comment within the code change. Gerrit Line The line number in the file at which the comment is added. Gerrit Message The text of the review comment. Gerrit Security-related Whether the review comment is security-related (i.e., Yes or No). Labelling Security defect type The type of the security defect identified in the comment. Labelling Consequence The Consequence of the security defect. Extraction Resolution Evidence The information about where the identified security defect was resolved in the code Extraction <strong>2. RQ2</strong>: <strong>Extracted data for RQ2.mx22</strong> The RQ2 folder contains the extracted data of 539 security-related review comments in <strong>Extracted data for RQ2.mx22</strong>, which was encoded and analyzed by the MAXQDA tool, investigating the treatment of security defects by developers and reviewers to answer RQ2. <strong>3. RQ3</strong>: <strong>Extracted data for RQ3.mx22</strong> The RQ3 folder contains the extracted data of 161 review comments in which identified security defects were not resolved by developers in <strong>Extracted data for RQ3.mx22</strong>. which was also encoded and analyzed by the MAXQDA tool, exploring the causes of not resolving security defects to answer RQ3. <strong>Note</strong>: The mx22 can be opened by MAXQDA 22, which are available at https://www.maxqda.com/ for download. You may also use the free trial version of MAXQDA 2022, which is available at https://www.maxqda.com/trial for download.本数据集对应论文《Security Defect Detection via Code Review: A Study of the OpenStack and Qt Communities》，包含提取所得数据与实验结果。本数据集包含以下三个文件夹： 1. **RQ1**：包含《Security defect in Nova.xlsx》《Security defect in Neutron.xlsx》《Security defect in Qt Base.xlsx》《Security defect in Qt Creator.xlsx》四个文件。RQ1文件夹对应四个研究项目（即OpenStack生态的Nova、Neutron组件，以及Qt生态的Qt Base与Qt Creator产品），共收录539条与安全相关的代码评审评论，所有评论均由评审人员标记出安全缺陷，均通过关键词检索后经人工标注获得，相关安全缺陷类型用于回答研究问题RQ1。 **如何读取RQ1中的Microsoft Excel（MS Excel）文件**：本文件夹内的4个Microsoft Excel文件均包含2017年至2022年共6个年度的工作表，每个工作表含10列数据项，其中后4列数据项用于本研究回答各研究问题。下文列出各数据项的详细信息： - **Keyword（关键词）**：评论对应的检索关键词，来源为基于关键词的检索； - **Code_change_id**：评论所属的代码变更ID，来源为Gerrit（代码评审平台）； - **File**：添加评论的代码文件，来源为Gerrit； - **Patchset**：代码变更中评论所属的补丁集，来源为Gerrit； - **Line**：评论所在代码文件的行号，来源为Gerrit； - **Message**：评审评论的文本内容，来源为Gerrit； - **Security-related**：评审评论是否与安全相关（是/否），来源为人工标注； - **Security defect type**：评论中识别出的安全缺陷类型，来源为人工标注； - **Consequence**：该安全缺陷的影响后果，来源为人工标注； - **Resolution Evidence**：已识别安全缺陷在代码中被修复的相关信息，来源为数据提取。 2. **RQ2**：包含文件《Extracted data for RQ2.mx22》。RQ2文件夹存储了539条安全相关评审评论的提取数据，该文件已通过MAXQDA工具进行编码与分析，用于探究开发者与评审人员对安全缺陷的处理流程，以回答研究问题RQ2。 3. **RQ3**：包含文件《Extracted data for RQ3.mx22》。RQ3文件夹存储了161条经识别但未被开发者修复的评审评论的提取数据，该文件同样通过MAXQDA工具进行编码与分析，用于探究安全缺陷未被修复的原因，以回答研究问题RQ3。 **注意**：mx22格式文件可通过MAXQDA 22打开，下载地址为https://www.maxqda.com/；也可使用MAXQDA 2022免费试用版，下载地址为https://www.maxqda.com/trial。