Making research evaluation more transparent: Aligning research philosophy, institutional values, and reporting

There is a growing interest in changing the culture of psychology to improve the quality of our science. At the root of this interest is concern over the reproducibility of key findings. A variety of large-scale replication attempts have revealed that a number of previously published effects cannot be reproduced, while other analyses indicate that the published literature is rife with underpowered studies and publication bias. These revelations suggest that it is time to change how psychological science is carried out and increase transparency of reporting. We argue change will be slow until institutions adopt new procedures for evaluating scholarly activity. We consider three actions that individuals and departments can take to facilitate change throughout psychological science. These three actions are the development of individualized research philosophy statements, the creation of an annotated curriculum vitae to improve the transparency of scholarly reporting, and the use of a formal evaluative system that explicitly captures behaviors that support reproducibility. Our recommendations build on proposals for open science by enabling researchers to have a voice in articulating (and contextualizing) how they would like their work to be evaluated and by providing a mechanism for more detailed and transparent reporting of scholarly activities.

心理学领域的文化变革日益受到关注，旨在提升科学研究的质量。此兴趣的根源在于对关键发现可重复性的担忧。众多大规模的复制尝试揭示，许多先前发表的效应无法重现，而其他分析表明，已发表的文献充斥着低效能研究和发表偏差。这些发现表明，是时候改变心理学科学的研究方法，并提高报告的透明度。我们认为，直到机构采用新的评估学术活动的程序，变革将进展缓慢。我们认为，个人和系所可以采取三项行动，以促进心理学科学领域的整体变革。这三项行动包括制定个性化的研究哲学声明，创建标注的个人履历，以增强学术报告的透明度，以及采用一种正式的评估体系，明确捕捉支持可重复性的行为。我们的建议建立在开放科学的提案之上，通过使研究人员能够在阐述（并情境化）他们希望自己的工作如何被评估方面发声，并提供了更详细和透明地报告学术活动的机制。