Impact of Software Engineering Research in Practice: A Patent and Author Survey Analysis

Dataset of the research paper: Impact of Software Engineering Research in Practice: A Patent and Author Survey Analysis

Existing work on the practical impact of software engineering (SE) research examines industrial relevance rather than adoption of study results, hence the question of how results have been practically applied remains open. To answer this and investigate the outcomes of impactful research, we performed a quantitative and qualitative analysis of 4 354 SE patents citing 1 690 SE papers published in four leading SE venues between 1975–2017. Moreover, we conducted a survey on 475 authors of 593 top-cited and awarded publications, achieving 26% response rate. Overall, researchers have equipped practitioners with various tools, processes, and methods, and improved many existing products. SE practice values knowledge-seeking research and is impacted by diverse cross-disciplinary SE areas. Practitioner-oriented publication venues appear more impactful than researcher-oriented ones, while industry-related tracks in conferences could enhance their impact. Some research works did not reach a wide footprint due to limited funding resources or unfavorable cost-benefit trade-off of the proposed solutions. The need for higher SE research funding could be corroborated through a dedicated empirical study. In general, the assessment of impact is subject to its definition. Therefore, academia and industry could jointly agree on a formal description to set a common ground for subsequent research on the topic.

The following data files are included.

./fields:

engi-fields.csv: Publication and PhD dissertation counts of main engineering branches

engi-fields-queries.txt: Queries applied to Elsevier's Scopus and Open Access Theses and Dissertations databases to retrieve the publication and dissertation counts

./patents:

sample-se-references-verified.csv: Manual verification of a random sample of references by software engineering (SE) patents to SE papers

se-cpc.tsv: Manually-identified SE-related Cooperative Patent Classification (CPC) categories

se-references-in-patents.csv: SE references made by SE patents to SE papers

./patents/litigation:

case-values.csv: Manually-retrieved litigation damages of citing SE patents

lit-per-paper.csv: Litigation cases of citing SE patents

./patents/maintenance:

maint-code-fee-mapping.csv: Mapping of patent maintenance fee codes to their fee values

maint-fees.csv: Fee values of maintenance fee codes

maint-per-paper.csv: Maintenance fee events of citing SE patents

./patents/reports:

lit-sum-per-paper.csv: Counts and total damages of litigation cases of patent-cited SE papers

maint-sum-per-paper.csv: Counts and total values of maintenance fee events of patent-cited SE papers

patent-ref-counts.csv: SE patent citation counts of patent-cited SE papers

./survey:

emse-top.csv: Most-cited papers of the Empirical Software Engineering (EMSE) journal

icse-bp.csv: Distinguished papers of the International Conference of Software Engineering (ICSE)

icse-mip.csv: Most influential ICSE papers

icse-top.csv: Most-cited ICSE papers

survey-questionnaire-emse.pdf: The EMSE survey questionnaire

survey-questionnaire.pdf: The ICSE, TSE, and TOSEM survey questionnaire

survey-responses.csv: The anonymized survey responses

tosem-top.csv: Most-cited papers of the ACM Transactions on Software Engineering and Methodology (TOSEM)

tse-top.csv: Most-cited papers of the IEEE Transactions on Software Engineering (TSE)

./survey/manual-coding:

feedback.txt: Manual coding of survey feedback

practical-impact.csv: Manual coding of responses about practical impact of work

practical-impact-lack.csv: Manual coding of responses about lack of practical impact

research-methods.csv: Manual coding of additional research methods of surveyed papers

state-of-practice.csv: Manual coding of responses about changes in state of practice

./venues:

se-venues.csv: Top SE venues according to Google Scholar Metrics

se-venues-impact.csv: SE patent citations and patent-based impact factors of SE venues

se-venues-scopus-queries.txt: Queries applied to Scopus to retrieve the publication counts of the SE venues