Gaussian Variational Approximation for Ordinal Data with Crossed Random Effects

We consider large scale ordinal response models with crossed random effects. We use a Gaussian variational approximation simplifying the associated optimization problem using the delta method, and other approximations to calculate both point estimates and standard errors whilst maintaining computational scalability. We apply our methodology on large scale recommender systems and are able to fit the proposed model with millions of observations within minutes on a standard laptop computer. Experiments show that estimation using this methodology scales close to linearly, and a factor better than the Penalized Quasi-Likelihood (PQL), with the number of observations, while maintaining an accuracy close to the Laplace Approximation (LA) with only slightly more underestimated variance parameters compared to LA in these large-scale settings. Our method thereby enables estimation of accurate, large scale ordinal models with crossed random effects.

我们研究具有交叉随机效应的大规模有序响应模型。我们采用高斯变分近似（Gaussian variational approximation），通过delta方法（delta method）简化相关优化问题，并结合其他近似手段计算点估计与标准误差，同时保持计算可扩展性。我们将该方法应用于大规模推荐系统，能够在标准笔记本电脑上以数分钟时间拟合包含数百万观测值的所提模型。实验表明，该方法的估计效率随观测值数量增加接近线性扩展，较惩罚拟似然法（Penalized Quasi-Likelihood, PQL）提升一个数量级；同时，其精度与拉普拉斯近似（Laplace Approximation, LA）相近，仅在方差参数估计上较LA略有低估（在大规模场景下）。该方法因此能够实现对具有交叉随机效应的准确、大规模有序模型的估计。