A review of expert group-based science advisory processes in Canada

Some of the most authoritative science advice comes from groups of experts operating under systematic advisory methods. In this paper, we compiled publicly available information on 676 science advisory processes conducted over a 28-year period by five Canadian science advisory institutions (i.e. the Canadian Science Advisory Secretariat, the National Advisory Council on Immunization, the Canadian Council of Academies, the Royal Society of Canada, and the Committee on the Status of Endangered Wildlife in Canada) and used these data to explore how these institutions operate. Despite common objectives (i.e. the delivery of sound scientific advice), we found considerable variation among institutions, including in the number of experts involved in developing and reviewing advice, the length of resulting science advice documents, and the time required for individual science advisory processes to be completed. In general, the institutions have become more transparent over time, driven primarily by disclosing more information on the experts involved. Additionally, we found that science advice reports have become lengthier, and the delivery of science advice now takes considerably more time for most institutions. We discuss these findings in the context of recent criticisms of expert group-based science advisory processes and suggest there may be trade-offs associated with emphasizing different science advice principles.

Methods
We collected basic information related to science advisory processes coordinated under the five different expert group-based science advice institutions described above using manual data extraction, and using the rvest (Wickham, 2024) and pdftools (Ooms, 2024) packages in R version 4.4.1 (R Core Team, 2021). The data collected included:

(i)             Title – the title of the science advice report produced as output of the advisory process;

(ii)            Publication Date – the date that the science advice report was publicly released;

(iii)           Advisory Period – the number of days between the start of the science advisory process and the publication date of the science advice report;

(iv)           Number of Authors – the number of experts involved in authoring the science advice report;

(v)            Number of Reviewers – the number of experts involved in reviewing and providing input on materials for the purpose of improving the science advice report and refining the final advice;

(vi)           Report Length – the number of pages in the science advice report, and;

(vii)          URL – a web link to the science advice report.

We compiled data between May 1 and August 9, 2024, from publicly available sources and spanned all years for which science advice reports were available (that is, for each institution, we included reports that stretched as far back in time as were available). Because of the large number of science advice reports available for CSAS (n = 1380) and COSEWIC (n = 1059), we randomly selected 10 reports per institution per calendar year to reduce the scope of our review while maintaining coverage over time. For other science advice institutions, we included all available reports. In total, we compiled information for 676 science advice reports produced between 1996 and 2024. Further details related to data collection are included in the Electronic Supplementary Material of the accompanying manuscript.