Estimating the Reproducibility of Psychological Science

Reproducibility is a defining feature of science, but the extent to which it characterizes current research is unknown. We conducted replications of 100 experimental and correlational studies published in three psychology journals using high-powered designs and original materials when available. Replication effects were half the magnitude of original effects, representing a substantial decline. Ninety-seven percent of original studies had statistically significant results. Thirty-six percent of replications had statistically significant results; 47% of original effect sizes were in the 95% confidence interval of the replication effect size; 39% of effects were subjectively rated to have replicated the original result; and if no bias in original results is assumed, combining original and replication results left 68% with statistically significant effects. Correlational tests suggest that replication success was better predicted by the strength of original evidence than by characteristics of the original and replication teams.

可重复性是科学的显著特征，但其对当前研究的表征程度尚不明确。我们采用高强度的设计和当可获得时使用原始材料，对发表在三家心理学期刊上的100项实验性和相关性研究进行了重复性研究。重复性效应仅为原始效应的一半，显示出显著的下降。97%的原研究具有统计学上显著的结果。36%的重复性研究具有统计学上显著的结果；47%的原效应量位于重复性效应量的95%置信区间内；39%的效应被主观评价为复制了原始结果；如果假设原始结果无偏差，则结合原始和重复性研究结果，仍有68%具有统计学上显著的效果。相关性测试表明，重复性成功的预测效果更多地取决于原始证据的强度，而非原始和重复性团队的特征。