Parallel variational Bayes for large datasets with an application to generalized linear mixed models

The article develops a hybrid Variational Bayes algorithm that combines the mean-field and stochastic linear regression fixed-form Variational Bayes methods. The new estimation algorithm can be used to approximate any posterior without relying on conjugate priors. We propose a divide and recombine strategy for the analysis of large datasets, which partitions a large dataset into smaller subsets and then combines the variational distributions that have been learnt in parallel on each separate subset using the hybrid Variational Bayes algorithm. We also describe an efficient model selection strategy using cross validation, which is straightforward to implement as a by-product of the parallel run. The proposed method is applied to fitting generalized linear mixed models. The computational efficiency of the parallel and hybrid Variational Bayes algorithm is demonstrated on several simulated and real datasets. Supplementary material for this article is available online.