Bayesian Federated Learning with Hamiltonian Monte Carlo: Algorithm and Theory

This work introduces a novel and efficient Bayesian federated learning algorithm, namely, the Federated Averaging stochastic Hamiltonian Monte Carlo (FA-HMC), for parameter estimation and uncertainty quantification. We establish rigorous convergence guarantees of FA-HMC on non-iid distributed datasets, under the strong convexity and Hessian smoothness assumptions. Our analysis investigates the effects of parameter space dimension, noise on gradients and momentum, and the frequency of communication (between the central node and local nodes) on the convergence and communication costs of FA-HMC. Beyond that, we establish the tightness of our analysis by showing that the convergence rate cannot be improved even for continuous FA-HMC process. Moreover, extensive empirical studies demonstrate that FA-HMC outperforms the existing Federated Averaging-Langevin Monte Carlo (FA-LD) algorithm. Supplementary materials for this article are available online.

本工作提出了一种新颖高效的贝叶斯联邦学习算法——联邦平均随机哈密顿蒙特卡洛（Federated Averaging stochastic Hamiltonian Monte Carlo，FA-HMC），用于参数估计与不确定性量化。我们在强凸性与海森（Hessian）光滑性假设下，针对非独立同分布（non-iid）分布式数据集，严格建立了FA-HMC的收敛性保障。本分析探究了参数空间维度、梯度与动量噪声、中心节点与本地节点间的通信频率，对FA-HMC的收敛性与通信开销的影响。除此之外，我们通过证明即便针对连续型FA-HMC过程，其收敛速率仍无法进一步提升，从而验证了本分析的紧致性。此外，大量实证研究表明，FA-HMC的性能优于现有联邦平均朗之万蒙特卡洛（Federated Averaging-Langevin Monte Carlo，FA-LD）算法。本文的补充材料可在线获取。