Measuring the aggregated impact of research: Establishing criteria for coding Translational Science Benefits Model data

Introduction: A promising approach to assessing research impact draws on the Translational Science Benefits Model (TSBM), an evaluation model that tracks the applied benefits of research in four domains: Clinical and Medical; Community and Public Health; Economic; and Policy and Legislative. However, standardized methods to verify TSBM benefit data, to aid in aggregating impact data within quantitative summaries, do not currently exist.

Methods: A panel of 11 topic experts participated in a modified Delphi process to establish content and face validity of a set of criteria for verifying qualitative TSBM data. Panelists were each affiliated with a Clinical Translational Science Award (CTSA) hub, including those at: Case Western Reserve University, Duke University School of Medicine, Medical University of South Carolina, University of Miami, University of Minnesota, University of New Mexico, and Washington University in St. Louis. Two survey rounds were completed by panelists, with a moderated discussion in between rounds to discuss criteria not reaching consensus to include. 

Data: The dataset available contains data from 3 surveys. One survey collected screening information about panelist's experiences with collecting, analyzing, and reporting TSBM data. Another survey collected responses from the first round of the delphi panel, which contained proposed criteria on all 30 TSBM benefits for panelists to respond if the criteria should be required. The last survey collected responses from the second round of the delphi panel, and was a condensed version of the survey with only the criteria on which the panelists had not previously reached consensus, as well as new criteria suggested by panelists in Round 1.