Data Supporting: Discovery Stack Pilot: Feasibility and Outcomes of a Scientist-Designed Peer Review Model

This repository contains the source data for the analyses and figures presented in the following manuscript: McGargill MA, Liu BC, Kuhns MS, Mucida D, Rauch I, Rodda LB, Koch MA, Gonzalez H, Cadwell K, Freedman TS, Scharschmidt TC, Sever R, Ordovas-Montanes J, Suliman S, Oberst A, Runnette B, Krummel MF. Discovery Stack Pilot: Feasibility and Outcomes of a Scientist-Designed Peer Review Model Separating Quality and Impact. bioRxiv 2025.10.31.685758. https://doi.org/10.1101/2025.10.31.685758 The dataset includes manuscript-level review scores and metadata, reviewer recruitment statistics, reviewer and author survey responses, and review-time measurements collected during the Discovery Stack Pilot study. Additional analyses and code are available at https://github.com/mmcgargi/Discovery-Stack-Analysis DSP_Paper_Info.xlsx — Number of Quality and Impact reviews per manuscript, version of manuscript reviewed in DSP (initial vs. revised submission), journal manuscript was submitted to, journal manuscript was published in, length of journal review, and average composite Quality and Impact scores. (Fig. 1B-C, Fig. 4E-F, Fig. 9A) DSP_Composite_Score.xlsx — Quality and Impact scores for each question, weights assigned to each question, all composite Quality and Impact scores, and composite Quality and Impact scores from reviewers who provided both (Matched). (Fig. 4A-F, Fig. S1C-F) DSP_QI_Correlation.xlsx — Results from correlations of Quality and Impact scores from all reviewers (Unmatched) and from reviewers who provided both (Matched). Discovery Stack Quality and Impact scores compared to submitted journal impact factor. (Fig. 5A-D) DSP_QI_Variance.xlsx — Cleaned list of composite scores used for analysis; SD, range, and IQR of each manuscript's Quality and Impact scores; subsampling-analysis summary. (Fig. 5A-D) DSP_Survey_Metric_Questions.xlsx — Summary tables of counts from survey questions regarding metrics. (Fig. 4G-J) DSP_Survey_All_Responses.xlsx — Survey responses from Quality/Impact reviewers, Impact-only reviewers, and Authors regarding the current platform tested and views on a future platform. Summary tables of selected responses appear in other files as noted. (Fig. 4G-J, Fig. 6, Fig. 7, Fig. 8, Fig. 9F,J, Fig. S3A-E, Fig. S3G-H, Fig. S4D-E, S4G-H, Fig. 10A-B) DSP_Survey_summary_Tables.xlsx — Summary tables of counts from survey questions on overall satisfaction, peer-improvement perception, Quality review components, Impact review components, beneficial aspects of the DSP platform, and challenges with the DSP platform. (Fig. 6A-D, Fig. 7A-C, Fig. 8A-D) DSP_Review_Time.xlsx — Duration of each review phase per manuscript, time of initial journal review, reviewer-reported completion time, individual reviewer times, and survey-reported times. (Fig. 9B-J) DSP_Survey_Future_Summary.xlsx — Summary tables of survey responses regarding priorities for a future platform. (Fig. 10A-B) DSP_Reviewer_Identity.xlsx — Number and percent of identified and anonymous reviewers per manuscript, reasons for reviewer anonymity, authors' perception of transparency, correlation of transparency with percent identified reviewers, perception of identity effect on score, and comparison of identified and anonymous reviewer scores. (Fig. S3A-M) DSP_recruitment.xlsx — Number of reviewers invited and number who accepted per manuscript; how authors and reviewers indicated they learned of the study; whether reviewers were DSP-enrolled, new to DSP, or new to DSP with email from a colleague (new to DSP + email); response rates of trainees vs. PIs; and survey respondents’ review experience. (Fig. S4A-H) DSP_Survey_openQuestion.xlsx — Responses to open-ended survey questions regarding priorities for change and priorities for a new platform. (Fig. S5A-C) DSP_Survey_Counts_New_Vs_old.xlsx – Counts of survey responses of participants new to the study (NEW) and those previously enrolled (DSP-enrolled) for each survey question, and a summary table of results from the Mann-Whitney U comparison. (Fig. S2) Quality_Impact_Reviewer_Survey_FIG.pdf – Questions on survey distributed to Quality/Impact reviewers at the completion of the pilot. Impact_Reviewer_Survey_FIG.pdf – Questions on survey distributed to Impact-only reviewers at the completion of the pilot. Author_Survey_FIG.pdf – Questions on survey distributed to authors at the completion of the pilot.