Statistical evaluation of research performance of young university scholars: A case study

Abstract The research performance of a small group of 49 young scholars, such as doctoral students, postdoctoral and junior researchers, working in different technical and scientific fields, was evaluated based on 11 types of research outputs. The scholars worked at a technical university in the fields of Civil Engineering, Ecology, Economics, Informatics, Materials Engineering, Mechanical Engineering, and Safety Engineering. Principal Component Analysis was used to statistically analyze the research outputs and its results were compared with factor and cluster analysis. The metrics of research productivity describing the types of research outputs included the number of papers, books and chapters published in books, the number of patents, utility models and function samples, and the number of research projects conducted. The metrics of citation impact included the number of citations and h-index. From these metrics – the variables – the principal component analysis extracted 4 main principal components. The 1st principal component characterized the cited publications in high-impact journals indexed by the Web of Science. The 2nd principal component represented the outputs of applied research and the 3rd and 4th principal components represented other kinds of publications. The results of the principal component analysis were compared with the hierarchical clustering using Ward’s method. The scatter plots of the principal component analysis and the Mahalanobis distances were calculated from the 4 main principal component scores, which allowed us to statistically evaluate the research performance of individual scholars. Using variance analysis, no influence of the field of research on the overall research performance was found. Unlike the statistical analysis of individual research metrics, the approach based on the principal component analysis can provide a complex view of the research systems.

摘要 本研究针对由49名青年学者组成的小型团队开展科研绩效评估，该团队涵盖博士生、博士后与初级研究员，均任职于某工科大学，研究领域覆盖土木工程、生态学、经济学、信息学、材料工程、机械工程及安全工程等不同技术与科学领域，评估基于11类科研产出展开。本研究采用主成分分析(Principal Component Analysis)对科研产出进行统计学分析，并将分析结果与因子分析、聚类分析的结果进行对比。表征科研产出类型的科研生产力指标包括：发表论文、图书及图书章节的数量，专利、实用新型与功能样本的数量，以及承担科研项目的数量；引文影响力指标则包括总被引频次与h指数。基于上述指标（即变量），主成分分析提取出4个核心主成分：第一主成分用于表征Web of Science收录的高影响力期刊发表论文的被引情况；第二主成分代表应用研究产出；第三、第四主成分则代表其他类型的科研出版物。将主成分分析结果与采用沃德法(Ward's method)的层级聚类分析结果进行对比。基于4个核心主成分得分计算主成分分析散点图与马氏距离，以此实现对个体学者科研绩效的统计学评估。通过方差分析(Analysis of Variance)未发现研究领域对整体科研绩效存在显著影响。相较于单一科研指标的统计分析，基于主成分分析的研究方法能够为科研系统提供更为全面的视角。