Making Research in Water Resources More Reproducible: ASCE's Reproducibility Review Program

This collection includes the abstract and slides from the talk titled \"Making Research in Water Resources More Reproducible: ASCE's Reproducibility Review Program\", presented by Jim Stagge at the 2023 Universities Council on Water Resources (UCOWR)/National Institutes for Water Resources (NIWR) Annual Water Resources Conference on June 15, 2023.

Abstract
There is strong agreement across scientific fields that replicable workflows are needed for computational modeling and that publishing digital research artifacts (workflow, code, data, output) alongside manuscripts increases the number of people who can access, learn by doing, and ultimately expand upon published research. This talk will highlight efforts by ASCE and the Journal of Water Resources Planning and Management (JWRPM) to implement a new reproducibility peer review system that operates in parallel with typical content reviews for submitted manuscripts. Authors opt in to the voluntary program during submission and share all research materials in a public repository. Reproducibility reviewers then use these materials to independently reproduce all figures, tables, and results, potentially making recommendations to improve the replicability/reproducibility of the submission. The program is voluntary, but offers several incentives in addition to the aforementioned benefits of increased research impact. Currently, papers that meet the reproducibility requirements are published in a special collection with open access fees waived by the journal. Additionally, papers are marked with a bronze or silver reproducibility badge to indicate making some artifacts publicly available (bronze) or replicated (silver). Finally, JWRPM has introduced new annual awards for reproducibility, separately awarding authors and reviewers for their efforts. Several years in, JWRPM has published 6 articles with a reproducibility badge, with 4 more in press, and an additional 17 in review or revision. Based on the success of this program, which began in 2020, ASCE is planning to expand it to journals outside water resources. Want to publish a paper with more impact, learn how to make your results more accessible to our community, or become a reproducibility reviewer? Attend this talk.