Examining Chi-Square Test Statistics Under Conditions of Large Model Size and Ordinal Data

This study examined the effect of model size on the chi-square test statistics obtained from ordinal factor analysis models. The performance of six robust chi-square test statistics were compared across various conditions, including number of observed variables (<i>p</i>), number of factors, sample size, model (mis)specification, number of categories, and threshold distribution. Results showed that the unweighted least squares (ULS) robust chi-square statistics generally outperform the diagonally weighted least squares (DWLS) robust chi-square statistics. The ULSM estimator performed the best overall. However, when fitting ordinal factor analysis models with a large number of observed variables and small sample size, the ULSM-based chi-square tests may yield empirical variances that are noticeably larger than the theoretical values and inflated Type I error rates. On the other hand, when the number of observed variables is very large, the mean- and variance-corrected chi-square test statistics (e.g., based on ULSMV and WLSMV) could produce empirical variances conspicuously smaller than the theoretical values and Type I error rates lower than the nominal level, and demonstrate lower power rates to reject misspecified models. Recommendations for applied researchers and future empirical studies involving large models are provided.

本研究考察了模型规模对从序数因子分析（ordinal factor analysis）模型中得到的卡方检验统计量的影响。研究对比了六种稳健卡方检验统计量在多种条件下的表现，包括观测变量数量（记为$p$）、因子数量、样本量、模型（误）设定、类别数以及阈值分布。结果显示，不加权最小二乘（unweighted least squares, ULS）稳健卡方统计量整体表现优于对角加权最小二乘（diagonally weighted least squares, DWLS）稳健卡方统计量，其中ULSM估计量的综合表现最佳。不过，当拟合含大量观测变量且样本量较小的序数因子分析模型时，基于ULSM的卡方检验所得到的经验方差会显著大于理论值，且一类错误率被高估。反之，当观测变量数量极多时，基于均值与方差校正的卡方检验统计量（例如基于ULSMV和WLSMV）所得到的经验方差会明显小于理论值，一类错误率低于名义水平，同时拒绝误设定模型的检验效力也更低。本研究为应用研究者以及涉及大型模型的后续实证研究提供了建议。