Neural Autoregressive Flows based Variational Bayes Model Averaging

Bayesian Model Averaging (BMA) enhances predictive performance by integrating over competing models, but its scalability is often limited by the computational burden of Markov Chain Monte Carlo (MCMC)–based posterior inference. Recent variational approaches such as VBMA (Kejzlar et al., 2023) offer scalability but rely on restrictive mean-field assumptions that fail to capture posterior dependencies, leading to suboptimal uncertainty quantification in complex settings. We propose Neural Autoregressive Flow Bayesian Model Averaging (NAF-BMA), a novel variational BMA framework that replaces the simple variational family with expressive neural autoregressive flows. This innovation enables NAF-BMA to model highly correlated, multimodal posterior structures with MCMC-level accuracy while retaining near–VBMA scalability. The method jointly estimates individual model evidences and posterior model probabilities within a unified optimization scheme, producing an optimal combined posterior over the entire model space. Designed as a general and modular framework requiring minimal model-specific derivations, NAF-BMA extends naturally to a broad class of Bayesian models. Across extensive simulation and real-data studies, it consistently outperforms VBMA and closely matches MCMC accuracy, establishing NAF-BMA as a flexible and scalable new paradigm for Bayesian model averaging.