Automatic Requirements Testability Analysis Dataset

This repository contains the data and results from the paper “Requirement testability measurement based on requirement smells”. We announce a public dataset of software requirements along with their quality issues to facilitate the development and evaluation of software requirement quality tools. The Automatic Requirement Testability Analyzer (ARTA) is a web application to measure requirement testability based on the requirement smells. It facilitates the requirement engineering quality assurance process and allows requirement engineers to manage the requirements of their projects. ARTA dataset is a collection of requirements along with their smells and degree of testability of each requirement. It has been used in the initial evaluation of ARTA testability measurement algorithms. To the best of our knowledge, there is no public dataset for developing and evaluating the requirements smell detector tools. Dataset Structure ARTA dataset contains 4752 software requirements extracted from 24 project documentation (in English). The ARTA dataset (release 1.0.0) is available as a collection of Microsoft Excel files. The Datasets directory contains five Excel files, described as follows: DS1.xlsx: This file contains 985 requirements from 6 projects and nine types of smells manually labeled for each requirement. DS2.xlsx: This file contains 1092 requirements from 8 projects and nine types of smells manually labeled for each requirement. DS3.xlsx: This file contains 1522 requirements from 6 projects and nine types of smells manually labeled for each requirement. DS4.xlsx: This file contains 1153 requirements from 4 projects and nine types of smells manually labeled for each requirement. SmellyWordsDictionary.xlsx: This file contains 1000 most frequent words in computer science domains, which is sorted based on their similarity. Four types of requirement smells are detected based on the words in the smelly words dictionary. For each word, the type of smell has been determined manually by experts. The DS1_Evaluation directory contains the result of different smell detection and testability measurement algorithms, described as follows: 001_dataset1kv1.xlsx (primary dataset): The same DS1 in the Datasets directory. This file contains 985 requirements from 6 projects and nine types of smells manually labeled for each requirement. 002_dataset1kv1_smell_frequency_with_testability.xlsx: This file contains the number of words of each requirement, the number of smelly words, number of smelly words according to the type of smell, degree of cleanness, and testability of each requirement, which has been computed based on “001_dataset1kv1.xlsx”. The requirements are in the same order in “001_dataset1kv1.xlsx”. 003_dataset1kv1_ARTA_result.xlsx: This file contains the result of automatically detected smells by ARTA, our proposed smell detector tool. The requirements are in the same order in “001_dataset1kv1.xlsx”. 004_dataset1kv1_ARTA_result_smell_frequency_with_testability.xlsx: This file contains the number of words of each requirement, the number of smelly words, number of smelly words according to the type of smell, degree of cleanness, and testability of each requirement, which has been computed based on “003_dataset1kv1_ARTA_result.xlsx”. The requirements are in the same order in “001_dataset1kv1.xlsx”. 005_dataset1kv1_Smella_result.xlsx: This file contains the result of automatically detected smells by Smella, another smell detector tool. The requirements are in the same order in “001_dataset1kv1.xlsx”. 006_dataset1kv1_Smella_result_smell_frequency_with_testability.xlsx: This file contains the number of words of each requirement, the number of smelly words, number of smelly words according to the type of smell, degree of cleanness, and testability of each requirement, which has been computed based on “005_dataset1kv1_Smella_result.xlsx”. The requirements are in the same order in “001_dataset1kv1.xlsx”.本仓库收录了论文《基于需求坏味（requirement smells）的需求可测试性度量》的配套数据与实验结果。为推动软件需求质量工具的开发与评估工作，我们公开了一批附带质量问题的软件需求数据集。自动需求可测试性分析器（Automatic Requirement Testability Analyzer，ARTA）是一款基于需求坏味实现需求可测试性度量的Web应用，可助力需求工程质量保障流程，并支持需求工程师对项目需求进行管理。 ARTA数据集收录了各类软件需求及其坏味标注，以及每条需求的可测试性等级，该数据集已被用于ARTA可测试性度量算法的初始评估。据我们所知，目前尚无用于开发与评估需求坏味检测工具的公开数据集。 数据集结构 ARTA数据集包含从24份英文项目文档中提取的4752条软件需求。ARTA数据集（版本1.0.0）以Microsoft Excel文件集合的形式发布。Datasets目录包含5个Excel文件，详情如下： 1. DS1.xlsx：该文件包含来自6个项目的985条需求，并为每条需求人工标注了9类需求坏味。 2. DS2.xlsx：该文件包含来自8个项目的1092条需求，并为每条需求人工标注了9类需求坏味。 3. DS3.xlsx：该文件包含来自6个项目的1522条需求，并为每条需求人工标注了9类需求坏味。 4. DS4.xlsx：该文件包含来自4个项目的1153条需求，并为每条需求人工标注了9类需求坏味。 5. SmellyWordsDictionary.xlsx：该文件收录了计算机科学领域中出现频率最高的1000个词汇，按词汇相似度排序。我们基于该坏味词典中的词汇检测4类需求坏味，每个词汇对应的坏味类型均由领域专家人工标定。 DS1_Evaluation目录包含各类坏味检测与可测试性度量算法的实验结果，详情如下： 1. 001_dataset1kv1.xlsx（主数据集）：与Datasets目录中的DS1完全一致，该文件包含来自6个项目的985条需求，并为每条需求人工标注了9类需求坏味。 2. 002_dataset1kv1_smell_frequency_with_testability.xlsx：该文件基于"001_dataset1kv1.xlsx"计算生成，包含每条需求的单词数、坏味词数量、按坏味类型划分的坏味词数量、洁净度等级以及可测试性等级，需求顺序与"001_dataset1kv1.xlsx"保持一致。 3. 003_dataset1kv1_ARTA_result.xlsx：该文件包含ARTA（我们提出的需求坏味检测工具）自动检测的坏味结果，需求顺序与"001_dataset1kv1.xlsx"保持一致。 4. 004_dataset1kv1_ARTA_result_smell_frequency_with_testability.xlsx：该文件基于"003_dataset1kv1_ARTA_result.xlsx"计算生成，包含每条需求的单词数、坏味词数量、按坏味类型划分的坏味词数量、洁净度等级以及可测试性等级，需求顺序与"001_dataset1kv1.xlsx"保持一致。 5. 005_dataset1kv1_Smella_result.xlsx：该文件包含另一款需求坏味检测工具Smella自动检测的坏味结果，需求顺序与"001_dataset1kv1.xlsx"保持一致。 6. 006_dataset1kv1_Smella_result_smell_frequency_with_testability.xlsx：该文件基于"005_dataset1kv1_Smella_result.xlsx"计算生成，包含每条需求的单词数、坏味词数量、按坏味类型划分的坏味词数量、洁净度等级以及可测试性等级，需求顺序与"001_dataset1kv1.xlsx"保持一致。