Rating versus ranking in a Delphi survey: a randomized controlled trial

The Delphi technique has steeply grown in popularity in health research as a structured approach to group communication process. Rating and ranking are two different procedures commonly used to quantify participants’ opinions in Delphi surveys. Little is known about the effect of assessment procedure on the outcome of the Delphi process, questionnaire completion time, and evaluation of task difficulty. This dataset pertains to a randomized controlled parallel group trial that was embedded in a three-round online Delphi survey to compare rating and ranking. After an “open” first round, primary care patients, trained patient partners, and primary care clinicians (n=36) from seven primary care practices in Quebec, Canada, were allocated 1:1 to a rating or ranking assessment group for the remainder of the study by stratified permuted block randomization, with strata based on participants’ gender and status. Items achieving an initial consensus level ≥66.6% in each study group during round 2 were reassessed during the final (third) round. The dataset related to the main Delphi study, in which the results from the rating and ranking groups were combined and differences between patient and clinician panelists were explored, is available here: https://doi.org/10.6084/m9.figshare.20110280.v1. The study was approved by the University of Montreal Hospital Research Centre’s research ethics committee (project number 17.305). Participant recruitment and data collection took place over a one-year period, from November 2019 to November 2020. <br> Please see the README_file for more information about the variables in the dataset. <br> <br>