Analysis of Repeatability and Reproducibility Studies With Ordinal Measurements

A Bayesian inferential approach with a noninformative prior is introduced to analyze ordinal repeatability and reproducibility (R&R) data using the De Mast–Van Wieringen model. This approach is extended with a weakly informative prior and random effects to allow for the consideration of a population of raters and prediction of a new rater. This random-effects approach is also shown to result in partial pooling of estimates across raters. In addition, match-probability-based measures to decompose ordinal R&R study data into contributions due to repeatability and due to reproducibility are defined. All extensions involving Bayesian inference (for fixed or random effects) and measures are illustrated on real and simulated ordinal R&R study data and are applicable in business and industry settings. This methodology can be implemented using the supplemental R package ordinalRR available from CRAN. Additional supplementary material for this article is available online.