Replication Package of Deep Learning and Data Augmentation for Detecting Self-Admitted Technical Debt
收藏资源简介：
Self-Admitted Technical Debt (SATD) refers to circumstances where developers use code comments, issues, pull requests, or other textual artifacts to explain why the existing implementation is not optimal. Past research in detecting SATD has focused on either identifying SATD (classifying SATD instances as SATD or not) or categorizing SATD (labeling instances as SATD that pertain to requirements, design, code, test, etc.). However, the performance of such approaches remains suboptimal, particularly when dealing with specific types of SATD, such as test and requirement debt. This is mostly because the used datasets are extremely imbalanced. In this study, we utilize a data augmentation strategy to address the problem of imbalanced data. We also employ a two-step approach to identify and categorize SATD on various datasets derived from different artifacts. Based on earlier research, a deep learning architecture called BiLSTM is utilized for the binary identification of SATD. The BERT architecture is then utilized to categorize different types of SATD. We provide the dataset of balanced classes as a contribution for future SATD researchers, and we also show that the performance of SATD identification and categorization using deep learning and our two-step approach is significantly better than baseline approaches. Therefore, to showcase the effectiveness of our approach, we compared it against several existing approaches: Natural Language Processing (NLP) and Matches task Annotation Tags (MAT) [Github] eXtreme Gradient Boosting+Synthetic Minority Oversampling Technique (XGBoost+SMOTE) [Figshare] eXtreme Gradient Boosting+Easy Data Augmentation (XGBoost+EDA) [Github] MT-Text-CNN [Github] Structure of the Replication Package: In accordance with the original dataset, the dataset comprises four distinct CSV files delineated by the artifacts under consideration in this study. Each CSV file encompasses a text column and a class, which indicate classifications denoting specific types of SATD, namely code/design debt (C/D), documentation debt (DOC), test debt (TES), and requirement debt (REQ) or Not-SATD. ├── SATD Keywords │ ├── Keywords based on Source of Artifacts │ │ ├── Code comment.txt │ │ ├── Commit message.txt │ │ ├── Issue section.txt │ │ └── Pull section.txt │ ├── Keywords based on Types of SATD │ │ ├── code-design debt.txt │ │ ├── documentation debt.txt │ │ ├── requirement debt.txt │ │ └── test debt.txt ├── src │ ├── bert.py │ ├── bilstm.py │ └── preprocessing.py ├── data-augmentation-code_comments.csv ├── data-augmentation-commit_messages.csv ├── data-augmentation-issues.csv ├── data-augmentation-pull_requests.csv └── Supplementary Material.docx Requirements: glove nltk transformers torch tensorflow keras langdetect inflect inflection Project sources for each artifact are as follows: Source code comment Issue section Pull section Commit message ant argouml columba emf hibernate jedit jfreechart jmeter jruby squirrel camel chromium gerrit hadoop hbase impala thrift accumulo activemq activemq-artemis airflow ambari apisix apisix-dashboard arrow attic-apex-core attic-apex-malhar attic-stratos avro beam bigtop bookkeeper brooklyn-server calcite camel camel-k camel-quarkus camel-website carbondata cassandra cloudstack commons-lang couchdb cxf daffodil drill druid dubbo echarts fineract flink fluo geode geode-native gobblin griffin groovy guacamole-client hadoop hawq hbase helix hive hudi iceberg ignite incubator-brooklyn incubator-dolphinscheduler incubator-doris incubator-heron incubator-hop incubator-mxnet incubator-pagespeed-ngx incubator-pinot incubator-weex infrastructure-puppet jena jmeter kafka karaf kylin lucene-solr madlib myfaces-tobago netbeans netbeans-website nifi nifi-minifi-cpp nutch openwhisk openwhisk-wskdeploy orc ozone parquet-mr phoenix pulsar qpid-dispatch reef rocketmq samza servicecomb-java-chassis shardingsphere shardingsphere-elasticjob skywalking spark storm streams superset systemds tajo thrift tinkerpop tomee trafficcontrol trafficserver trafodion tvm usergrid zeppelin zookeeper accumulo activemq activemq-artemis airflow ambari apisix apisix-dashboard arrow attic-apex-core attic-apex-malhar attic-stratos avro beam bigtop bookkeeper brooklyn-server calcite camel camel-k camel-quarkus camel-website carbondata cassandra cloudstack commons-lang couchdb cxf daffodil drill druid dubbo echarts fineract flink fluo geode geode-native gobblin griffin groovy guacamole-client hadoop hawq hbase helix hive hudi iceberg ignite incubator-brooklyn incubator-dolphinscheduler incubator-doris incubator-heron incubator-hop incubator-mxnet incubator-pagespeed-ngx incubator-pinot incubator-weex infrastructure-puppet jena jmeter kafka karaf kylin lucene-solr madlib myfaces-tobago netbeans netbeans-website nifi nifi-minifi-cpp nutch openwhisk openwhisk-wskdeploy orc ozone parquet-mr phoenix pulsar qpid-dispatch reef rocketmq samza servicecomb-java-chassis shardingsphere shardingsphere-elasticjob skywalking spark storm streams superset systemds tajo thrift tinkerpop tomee trafficcontrol trafficserver trafodion tvm usergrid zeppelin zookeeper This dataset has undergone a data augmentation process using the AugGPT technique. Meanwhile, the original dataset can be downloaded via the following link: https://github.com/yikun-li/satd-different-sources-data
自我承认的技术债务（Self-Admitted Technical Debt, SATD）指开发者通过代码注释、问题工单、拉取请求或其他文本制品，说明现有实现并非最优的场景。过往针对SATD检测的研究，要么聚焦于SATD识别（将实例分类为SATD或非SATD），要么聚焦于SATD分类（将实例标注为对应需求、设计、代码、测试等类别的SATD）。然而此类方法的性能仍未达最优，尤其是在处理特定类型的SATD（如测试债务与需求债务）时，这主要是因为所用数据集存在严重的类别不平衡问题。 本研究采用数据增强策略以解决类别不平衡问题，并提出两步法，针对源自不同文本制品的各类数据集完成SATD的识别与分类。基于前期研究，我们采用双向长短期记忆网络（BiLSTM）架构完成SATD的二元识别任务，随后采用BERT架构对不同类型的SATD进行分类。我们将平衡后的类别数据集作为开源贡献提供给后续SATD研究者，同时证明了结合深度学习与我们提出的两步法的SATD识别与分类性能，显著优于基线方法。 为验证我们方法的有效性，我们与多种现有方法进行了对比：自然语言处理（Natural Language Processing, NLP）与匹配任务标注标签（Matches task Annotation Tags, MAT）[GitHub]、极限梯度提升+合成少数类过采样技术（XGBoost+SMOTE）[Figshare]、极限梯度提升+简易数据增强（XGBoost+EDA）[GitHub]、MT-Text-CNN [GitHub]。 复制包结构：遵照原始数据集的划分规则，本数据集包含4个独立的CSV文件，对应本研究中所用的各类文本制品。每个CSV文件均包含文本列与类别列，类别用于标注特定类型的SATD，具体包括：代码/设计债务（C/D）、文档债务（DOC）、测试债务（TES）、需求债务（REQ）以及非SATD（Not-SATD）。 ├── SATD 关键词 │ ├── 基于制品来源的关键词 │ │ ├── 代码注释.txt │ │ ├── 提交信息.txt │ │ ├── 问题工单板块.txt │ │ └── 拉取请求板块.txt │ ├── 基于SATD类型的关键词 │ │ ├── 代码-设计债务.txt │ │ ├── 文档债务.txt │ │ ├── 需求债务.txt │ │ └── 测试债务.txt ├── src │ ├── bert.py │ ├── bilstm.py │ └── preprocessing.py ├── data-augmentation-code_comments.csv ├── data-augmentation-commit_messages.csv ├── data-augmentation-issues.csv ├── data-augmentation-pull_requests.csv └── 补充材料.docx 依赖要求：glove、nltk、transformers、torch、tensorflow、keras、langdetect、inflect、inflection 各项目源：各文本制品对应的项目源如下：源代码注释、问题工单、拉取请求、提交信息对应的项目包括：ant、argouml、columba、emf、hibernate、jedit、jfreechart、jmeter、jruby、squirrel、camel、chromium、gerrit、hadoop、hbase、impala、thrift、accumulo、activemq、activemq-artemis、airflow、ambari、apisix、apisix-dashboard、arrow、attic-apex-core、attic-apex-malhar、attic-stratos、avro、beam、bigtop、bookkeeper、brooklyn-server、calcite、camel、camel-k、camel-quarkus、camel-website、carbondata、cassandra、cloudstack、commons-lang、couchdb、cxf、daffodil、drill、druid、dubbo、echarts、fineract、flink、fluo、geode、geode-native、gobblin、griffin、groovy、guacamole-client、hadoop、hawq、hbase、helix、hive、hudi、iceberg、ignite、incubator-brooklyn、incubator-dolphinscheduler、incubator-doris、incubator-heron、incubator-hop、incubator-mxnet、incubator-pagespeed-ngx、incubator-pinot、incubator-weex、infrastructure-puppet、jena、jmeter、kafka、karaf、kylin、lucene-solr、madlib、myfaces-tobago、netbeans、netbeans-website、nifi、nifi-minifi-cpp、nutch、openwhisk、openwhisk-wskdeploy、orc、ozone、parquet-mr、phoenix、pulsar、qpid-dispatch、reef、rocketmq、samza、servicecomb-java-chassis、shardingsphere、shardingsphere-elasticjob、skywalking、spark、storm、streams、superset、systemds、tajo、thrift、tinkerpop、tomee、trafficcontrol、trafficserver、trafodion、tvm、usergrid、zeppelin、zookeeper（注：原文存在重复项目列表，未作修改）。 本数据集采用AugGPT技术完成了数据增强流程。同时，原始数据集可通过以下链接下载：https://github.com/yikun-li/satd-different-sources-data
