Machine Learning for Software Engineering: A Tertiary Study

Dataset of the research paper: <strong>Machine Learning for Software Engineering: A Tertiary Study</strong> Machine learning (ML) techniques increase the effectiveness of software engineering (SE) lifecycle activities. We systematically collected, quality-assessed, summarized, and categorized 39 reviews on ML for SE published between 2009–2020, covering 2,506 primary studies. The SE areas most tackled with ML are software testing and quality, while human-centered areas appear more challenging for ML. We propose a number of ML for SE research challenges and actions including: conducting further empirical validation and industrial studies on ML; reconsidering deficient SE methods; documenting and automating data collection and pipeline processes; reexamining how industrial practitioners distribute their proprietary data; and implementing incremental ML approaches. The following data and source files are included. <strong>review-protocol.md</strong>: The protocol employed in this tertiary study <strong>data/</strong> <strong> dl-search/</strong> <strong> input/</strong> <strong>acm_comput_surveys_overviews.bib</strong>: Surveys of ACM Computing Surveys journal <strong>acm_comput_surveys_overviews_titles.txt</strong>: Titles of surveys <strong>acm_comput_ml_surveys.bib</strong>: ML-related surveys of ACM Computing Surveys journal <strong>acm_comput_ml_surveys_titles.txt</strong>: Titles of ML-related surveys <strong>dl_search_queries.txt</strong>: Search queries applied to IEEE Xplore, ACM Digital Library, and Elsevier Scopus <strong>ml_keywords.txt</strong>: ML-related keywords extracted from ML-related survey titles and used in the search queries <strong>se_keywords.txt</strong>: SE-related keywords derived from the 15 SWEBOK Knowledge Areas (KAs—except for Computing Foundations, Mathematical Foundations, and Engineering Foundations) and used in the search queries <strong>secondary_studies_keywords.txt</strong>: Survey-related keywords composed of the 15 keywords introduced in the tertiary study on SLRs in SE by Kitchenham <em>et al.</em> (2010), and the survey titles, and used in the search queries <strong> output/</strong> <strong>acm/</strong> <strong>acm{1–9}.bib</strong>: Search results from ACM Digital Library <strong>ieee.csv</strong>: Search results from IEEE Xplore <strong>scopus_analyze_year.csv</strong>: Yearly distribution of ML and SE documents extracted from Scopus's Analyze search results page <strong>scopus.csv</strong>: Search results from Scopus <strong> study-selection/</strong> <strong>backward_snowballing.csv</strong>: Additional secondary studies found through the backward snowballing process <strong>cohen_kappa_agreement.csv</strong>: Inter-rater reliability of reviewers in study selection <strong>dl_search_results.csv</strong>: Aggregated search results of all three digital libraries <strong>study_selection_reviewer_{1–2}.csv</strong>: Divided search results assessed by reviewer 1 and 2, correspondingly, based on IC/EC <strong> quality-assessment/</strong> <strong>dare_assessment.csv</strong>: Quality assessment (QA) of selected secondary studies based on the Database of Abstracts of Reviews of Effects (DARE) criteria by York University, Centre for Reviews and Dissemination <strong>quality_accepted_studies.csv</strong>: Details of quality-accepted studies <strong>studies_for_review.bib</strong>: Bibliography details and QA scores of selected secondary studies <strong> data-extraction/</strong> <strong>further_research.csv</strong>: Recommendations for further research of quality-accepted studies <strong>knowledge_areas.csv</strong>: Classification of quality-accepted studies using the SWEBOK KAs and subareas <strong>ml_techniques.csv</strong>: Classification of the quality-accepted studies based on a four-axis ML classification scheme, along with extracted ML techniques employed in the studies <strong>primary_studies.csv</strong>: Details of reviewed primary studies by the quality-accepted secondary <strong>research_methods.csv</strong>: Citations of the research methods employed by the quality-accepted studies <strong>research_types_methods.csv</strong>: Research types and methods employed by the quality-accepted studies <strong>src/</strong> <strong>data-analysis.ipynb</strong>: Analysis of data extraction results (data preprocessing, top authors and institutions, study types, yearly distribution of publishers and QA scores) and creation of all figures included in the study <strong>scopus-year-analysis.ipynb</strong>: Yearly distribution of ML and SE publications retrieved from Elsevier Scopus <strong>study-selection-preprocessing.ipynb</strong>: Processing of digital library search results to conduct the inter-rater reliability estimation and study selection process