The effectiveness of journals as arbiters of scientific quality

Academic publishers purport to be arbiters of knowledge, aiming to publish studied that advance the frontiers of their research domain. Yet the effectiveness of journal editors at identifying novel and important research is generally unknown, in part because of the confidential nature of the editorial and peer-review process. Using questionnaires, we evaluated the degree to which journals are effective arbiters of scientific impact in the domain of Ecology, quantified by three key criteria. First, journals discriminated against low-impact manuscripts: the probability of rejection increased as the number of citations gained by the published paper decreased. Second, journals were more likely to publish high-impact manuscripts (those that obtained citations in 90th percentile for their journal) than run-of-the-mill manuscripts; editors were only 23 and 41% as likely to reject an eventual high-impact paper (pre- versus post-review rejection) compared to a run-of-the-mill paper. Third, edito...