Scoping Review Data - Research Impact Assessment

This article draws on part of a larger project where we conducted a scoping review on research impact. We used the following search string on JSTOR, Scopus and Web of Science: ‘research impact’ or ‘impact of research’. The database search was conducted on 15 April 2023 without any restriction on the date. Only journal articles in English were included, while books, book chapters, reports, research notes, conference proceedings and editorials were excluded. Accessing books and book chapters presented some challenges, but the corpus was large enough to exclude these types of publications. The abstract and full-text screening phase was conducted by two researchers using the Rayyan© review tool.First, the abstracts were screened and only those that mentioned research impact and/or evaluation, as well as similar concepts such as useful research and research relevance, were included. Second, articles were full-text screened, and those that discussed research impact at length in their analysis, conclusion or discussion – not just in their introduction or literature review – were retained. A total of 1,270 articles were analysed and categorised into five groups depending on their main purposes: (1) analysing research impact – studies examining or assessing research impact through established tools and frameworks; (2) revealing dimensions of research impact – articles exploring practices and perspectives regarding impact; (3) proposing a new framework or tool – studies presenting new tools and frameworks for research impact; (4) conceptual or theoretical discussions – articles discussing conceptual and theoretical dimensions of impact and related concepts; and (5) services for research impact – studies focusing on impact-related services and institutional capacities.In this paper, which aims to investigate how impact assessment is addressed, we analyse the articles belonging to the ‘analysing research impact’ category as well as the articles that analyse impact while ‘proposing a new framework or tool’. Regarding the latter, we excluded the articles that propose a new framework or tool without analysing impact (87 articles out of 158). There are 822 articles in total: 751 articles analysing impact through established methods, frameworks and tools, and 71 articles that analyse impact through a new framework or tool. These articles were coded by publication date, discipline, impact type, impact approach and impact assessment method.The disciplines fields were coded according to the Scopus categorisation framework: social sciences, health sciences, life sciences, physical sciences and multidisciplinary. ‘Interdisciplinary’ was added as a category for articles in which the mentioned project or research was conducted with at least two disciplines mentioned above; the interdisciplinarity of an article was often explicitly stated by the authors or it was obvious that the research topic could not fit into only one category in the Scopus list.Impact type included academic impact (i.e. impact that occurs within academic settings), non-academic impact (i.e. impact beyond academia) and both. The impact approach was described as outcome-focused when it conceived impact as what happens after the research results are obtained. It was described as process-focused when it emphasised the impacts and activities during the research process. The outcome-focused approach emphasises all the outputs, outcomes and activities that follow research results (e.g. dissemination of research results through media), while the process-focused approach includes impact-generating mechanisms such as collaboration and co-production of knowledge and direct impacts of research processes and activities (e.g. impact of interviewing process on participants). Although dissemination of the research findings can be considered a process per se, we approach it as outcome since these activities took place after the results of the research were obtained. Additionally, academic collaboration is considered an outcome when it is assessed through co-authorship and identified as a process when explained beyond outputs.The impact assessment method included qualitative analysis, quantitative analysis and mixed analysis. Mixed analysis constitutes not only mixed methods but also a method or a framework with both qualitative and quantitative indicators.<br><br>