Gibbs Sampling using Anti-correlation Gaussian Data Augmentation, with Applications to L1-ball-type Models

L1-ball-type priors are a recent generalization of the spike-and-slab priors. By transforming a continuous precursor distribution to the L1-ball boundary, it induces exact zeros with positive prior and posterior probabilities. With great flexibility in choosing the precursor and threshold distributions, we can easily specify models under structured sparsity, such as those with dependent probability for zeros and smoothness among the non-zeros. Motivated to significantly accelerate the posterior computation, we propose a new data augmentation that leads to a fast block Gibbs sampling algorithm. The latent variable, named anti-correlation Gaussian, cancels out the quadratic exponent term in the latent Gaussian distribution, making the parameters of interest conditionally independent so that they can be updated in a block. Compared to existing algorithms such as the No-U-Turn sampler, the new blocked Gibbs sampler has a very low computing cost per iteration and shows rapid mixing of Markov chains. We establish the geometric ergodicity guarantee of the algorithm in linear models. Further, we show useful extensions of our algorithm for posterior estimation of general latent Gaussian models, such as those involving multivariate truncated Gaussian or latent Gaussian process.

L1球型先验（L1-ball-type priors）是尖峰-平板先验（spike-and-slab priors）的新近推广。通过将连续前驱分布转换至L1球边界，该先验可在兼具正先验与后验概率的前提下诱导出精确零点。得益于前驱分布与阈值分布的灵活选择，我们可便捷地指定结构化稀疏下的模型，例如具备零点相依概率与非零分量间平滑性的模型。为显著加速后验计算，我们提出一种全新的数据增广方法，由此得到快速分块吉布斯采样算法。该算法所引入的名为反相关高斯分布的隐变量，可抵消隐高斯分布中的二次指数项，使得待估参数满足条件独立，从而支持分块更新。相较于无回旋采样器（No-U-Turn sampler）等现有算法，该新型分块吉布斯采样器的单次迭代计算成本极低，且可实现马尔可夫链的快速混合。我们在线性模型框架下证明了该算法的几何遍历性保证。进一步地，我们展示了该算法在一般隐高斯模型后验估计中的实用扩展，例如涉及多元截断高斯分布或隐高斯过程的模型。