Transport Monte Carlo: High-Accuracy Posterior Approximation via Random Transport

In Bayesian applications, there is a huge interest in rapid and accurate estimation of the posterior distribution, particularly for high dimensional or hierarchical models. In this article, we propose to use optimization to solve for a joint distribution (random transport plan) between two random variables, θ from the posterior distribution and β from the simple multivariate uniform. Specifically, we obtain an approximate estimate of the conditional distribution Π(β|θ) as an infinite mixture of simple location-scale changes; applying the Bayes’ theorem, Π(θ|β) can be sampled as one of the reversed transforms from the uniform, with the weight proportional to the posterior density/mass function. This produces independent random samples with high approximation accuracy, as well as nice theoretical guarantees. Our method shows compelling advantages in performance and accuracy, compared to the state-of-the-art Markov chain Monte Carlo and approximations such as variational Bayes and normalizing flow. We illustrate this approach via several challenging applications, such as sampling from multi-modal distribution, estimating sparse signals in high dimension, and soft-thresholding of a graph with a prior on the degrees. Supplementary materials for this article, including the source code and additional comparison with popular alternative algorithms are available on the journal website.在贝叶斯应用领域，学界对快速且精准地估计后验分布（posterior distribution）有着强烈的研究需求，针对高维模型或分层模型的后验分布估计更是如此。本文提出通过优化方法求解两个随机变量间的联合分布，即随机运输规划（random transport plan），其中一个随机变量θ服从后验分布，另一个β服从简单的多元均匀分布（multivariate uniform）。具体而言，我们将条件分布Π(β|θ)的近似估计表示为简单位置-尺度变换的无限混合模型；结合贝叶斯定理（Bayes’ theorem），Π(θ|β)可通过对均匀分布进行反向变换采样得到，其权重与后验密度/质量函数成正比。该方法可生成具有高近似精度的独立随机样本，同时具备良好的理论保障。相较于当前主流的马尔可夫链蒙特卡洛（Markov chain Monte Carlo）以及变分贝叶斯（variational Bayes）、归一化流（normalizing flow）等近似推断方法，本文所提方法在性能与精度上展现出显著优势。我们通过多个具有挑战性的应用场景验证了该方法的有效性，例如从多模态分布（multi-modal distribution）中采样、高维稀疏信号（sparse signals）估计，以及对带有度数先验的图进行软阈值处理（soft-thresholding）。本文的补充材料（包括源代码以及与主流替代算法的额外对比实验）可在期刊官网获取。