Learning Coefficient Heterogeneity over Networks: A Distributed Spanning-Tree-Based Fused-Lasso Regression

Identifying the latent cluster structure based on model heterogeneity is a fundamental but challenging task arises in many machine learning applications. In this paper, we study the clustered coefficient regression problem in the distributed network systems, where the data are locally collected and held by nodes. Our work aims to improve the regression estimation efficiency by aggregating the neighbors’ information while also identifying the cluster membership for nodes. To achieve efficient estimation and clustering, we develop a distributed spanning-tree-based fused-lasso regression (DTFLR) approach. In particular, we propose an adaptive spanning-tree-based fusion penalty for the low-complexity clustered coefficient regression. We show that our proposed estimator satisfies statistical oracle properties. Additionally, to solve the problem parallelly, we design a distributed generalized alternating direction method of multiplier algorithm, which has a simple node-based implementation scheme and enjoys a linear convergence rate. Collectively, our results in this paper contribute to the theories of low-complexity clustered coefficient regression and distributed optimization over networks. Thorough numerical experiments and real-world data analysis are conducted to verify our theoretical results, which show that our approach outperforms existing works in terms of estimation accuracy, computation speed, and communication costs.

基于模型异质性（model heterogeneity）识别潜在簇结构，是广泛存在于诸多机器学习应用中的一项基础且极具挑战性的任务。本文针对分布式网络系统中的聚类系数回归（clustered coefficient regression）问题展开研究，该系统内的数据由各节点本地采集并存储。本研究旨在通过聚合邻节点信息提升回归估计效率，同时完成对节点簇成员的识别。为实现高效估计与聚类，我们提出了一种基于分布式生成树（spanning-tree）的融合套索回归（distributed spanning-tree-based fused-lasso regression，DTFLR）方法。具体而言，我们针对低复杂度聚类系数回归问题，提出了一种自适应生成树融合惩罚项。我们证明，所提出的估计量满足统计Oracle性质（statistical oracle properties）。此外，为实现并行求解，我们设计了分布式广义交替方向乘子法（distributed generalized alternating direction method of multipliers）算法，该算法具备简洁的基于节点的实现方案，且具有线性收敛速率。综上，本文的研究成果丰富了低复杂度聚类系数回归理论与网络分布式优化理论。我们通过大量数值实验与真实数据分析验证了理论结果，结果表明所提方法在估计精度、计算速度与通信开销方面均优于现有研究。