Agreement and reliability of statistical raters.

Background:Statistical models are valuable tools for interpreting complex relationships within health systems. These models rely on a framework of statistical assumptions that, when correctly addressed, enable valid inferences and conclusions. However, failure to appropriately address these assumptions can lead to flawed analyses, resulting in misleading conclusions and contributing to the adoption of ineffective or harmful treatments and poorer health outcomes. This study examines researchers’ understanding of the widely used linear regression model, focusing on assumptions, common misconceptions, and recommendations for improving research practices.Methods:One hundred papers were randomly sampled from the journal PLOS ONE, which used linear regression in the materials and methods section and were from the health and biomedical field in 2019. Two independent volunteer statisticians rated each paper for the reporting of linear regression assumptions. The prevalence of assumptions reported by authors was described using frequencies, percentages, and 95% confidence intervals. The agreement of statistical raters was assessed using Gwet’s statistic.Results:Of the 95 papers that met the inclusion and exclusion criteria, only 37% reported checking any linear regression assumptions, 22% reported checking one assumption, and no authors checked all assumptions. The biggest misconception was that the Y variable should be checked for normality, with only 5 of the 28 papers correctly checking the residuals for normality.Conclusion:The reporting of linear regression assumptions is alarmingly low. When assumptions are checked, the reporting is often inadequate or incorrectly checked. Addressing these issues requires a cultural shift in research practices, including improved statistical training, more rigorous journal review processes, and a broader understanding of regression as a unifying framework. Greater emphasis must be placed on evaluating model assumptions and their implications rather than the rote application of statistical methods. Careful consideration of assumptions helps improve the reliability of statistical conclusions, reducing the risk of misleading findings influencing clinical practice and potentially affecting patient outcomes.