Evaluating the Performance of R-Squared Measures in Multilevel Models

Multilevel Models (MLMs) have become a valuable tool in the behavioral and social sciences, providing a framework for analyzing clustered data structures commonly encountered in these fields. Unlike single-level regression, R2 measures in MLMs become more intricate due to the need to account for sources of variance at different levels. Recently, Rights and Sterba (2019) introduced an integrative framework of MLM R2 measures, providing a unifying approach to interpreting MLM R2 measures in relation to specific substantive questions. While this framework represents a valuable resource for applied research, the R2 measures have been defined in the population, and their performance across various conditions reflecting applied MLM practices remains unexplored. The present study evaluates the performance of the different MLM R2 measures as estimators of their population values through Monte Carlo simulations. Among other factors, we examined how the number of level-1 and level-2 predictors, cross-level interactions, and random slopes affect the accuracy of the corresponding MLM R2 measures. Results indicate that as the number of level-2 predictors increases, a greater number of clusters is required to ensure accurate estimates. The greater the number of level-1 predictors, cross-level interactions, and random slopes, increasing either the number of clusters or the number of observations per cluster leads to more accurate estimates.