Double-blind peer review affects reviewer ratings and editor decisions at an ecology journal

There is substantial evidence that systemic biases influence the scholarly
peer review process. Many scholars have advocated for double-blind peer
review (also known as double-anonymous review) to reduce these biases.
However, the effectiveness of double‐blind peer review in eliminating
biases is uncertain because few randomized trials have manipulated
blinding of author identities for journal submissions, and those that have
are generally small or provide few insights on how it influences reviewer
biases. In 2019, Functional Ecology began a large randomized trial, using
real manuscript submissions, to evaluate the various consequences of
shifting to double-blind peer review. Research papers submitted to the
journal were randomly assigned to be reviewed with author identities
blinded to reviewers (double‐blind review) or with authors identified to
reviewers (single-blind review). In this paper, we explore the effect of
blinding on the outcomes of peer review, examining reviewer ratings and
editorial decisions, and ask whether author gender and/or location mediate
the effects of review type. Double-blind review reduced the average
success of manuscripts in peer review; papers reviewed with author
identities blinded received on average lower ratings from reviewers and
were less likely to be invited for revision or resubmission. However, the
effect of review treatment varied with author location. Papers with first
authors residing in countries with a higher human development index (HDI)
and/or higher average English proficiency fared much better than those
from countries with a lower HDI and lower English proficiency, but only
when author identities were known to reviewers; outcomes were similar
between demographic groups when author identities were not known to
reviewers. Blinding author identities had no effect on gender differences
in reviewer ratings or editor decisions. Our data provide strong evidence
that authors from higher income and/or English-speaking countries receive
significant benefits (a large positive bias) to being identified to
reviewers during the peer review process, and that anonymizing
author-identities (e.g., double-blind review) reduces this bias, making
the peer review process more equitable. We suggest that offering optional
blinding of author identities, as some journals allow, is unlikely to
substantially reduce the biases that exist because authors from
higher-income and English-speaking countries are the least likely to
choose to be reviewed with their identity anonymized.