Variational Approximations for Generalized Linear Latent Variable Models

Generalized linear latent variable models (GLLVMs) are a powerful class of models for understanding the relationships among multiple, correlated responses. Estimation, however, presents a major challenge, as the marginal likelihood does not possess a closed form for nonnormal responses. We propose a variational approximation (VA) method for estimating GLLVMs. For the common cases of binary, ordinal, and overdispersed count data, we derive fully closed-form approximations to the marginal log-likelihood function in each case. Compared to other methods such as the expectation-maximization algorithm, estimation using VA is fast and straightforward to implement. Predictions of the latent variables and associated uncertainty estimates are also obtained as part of the estimation process. Simulations show that VA estimation performs similar to or better than some currently available methods, both at predicting the latent variables and estimating their corresponding coefficients. They also show that VA estimation offers dramatic reductions in computation time particularly if the number of correlated responses is large relative to the number of observational units. We apply the variational approach to two datasets, estimating GLLVMs to understanding the patterns of variation in youth gratitude and for constructing ordination plots in bird abundance data. R code for performing VA estimation of GLLVMs is available online. Supplementary materials for this article are available online.

广义线性潜变量模型（Generalized linear latent variable models，GLLVMs）是一类用于探究多变量关联响应间内在关系的性能优异的模型。然而，其参数估计面临重大挑战：对于非正态响应变量，边际似然（marginal likelihood）并无闭合形式（closed form）解。本文提出一种用于GLLVM参数估计的变分近似（variational approximation，VA）方法。针对二分类、有序分类以及过度分散计数数据这三类常见场景，本文分别推导了边际对数似然函数（marginal log-likelihood function）的完全闭合形式近似解。相较于期望最大化（expectation-maximization，EM）算法等其他方法，基于变分近似的估计方法计算速度更快，且实现流程更为简便直接。在估计过程中，还可同步得到潜变量的预测值及其对应的不确定性估计结果。仿真实验结果表明，无论是在潜变量预测还是对应系数估计方面，变分近似估计的性能均与现有部分方法相当，甚至更优。此外，仿真结果还显示，当关联响应变量的数量相较于观测单元数量较多时，变分近似估计可大幅缩短计算时长。本文将该变分方法应用于两个数据集：一是通过拟合GLLVMs探究青少年感恩情绪的变异模式，二是用于鸟类丰度数据的排序绘图。用于实现GLLVMs变分近似估计的R代码已公开于网络。本文的补充材料亦可在线获取。