Modeling Preferences: A Bayesian Mixture of Finite Mixtures for Rankings and Ratings

Rankings and ratings are commonly used to express preferences but provide distinct and complementary information. Rankings give ordinal and scale-free comparisons but lack granularity; ratings provide cardinal and granular assessments but may be highly subjective or inconsistent. Collecting and analyzing rankings and ratings jointly has not been performed until recently due to a lack of principled methods. In this work, we propose a flexible, joint statistical model for rankings and ratings—the Bradley-Terry-Luce-Binomial (BTL-Binomial). The model captures rater effects and preference heterogeneity, respectively, with judge-specific random effects and a latent class mixture framework where the number of classes is unknown a priori. We propose computationally-efficient estimation via a Bayesian mixture of finite mixtures (MFM) approach. Finally, we demonstrate statistical inference and decision-making based on rankings and ratings jointly through applications to real and simulated datasets in academic peer review. Supplementary materials for this article are available online, including a standardized description of the materials available for reproducing the work.