Unveil Linear Patterns of Dependence via K Regression Clustering

Clustering is a fundamental problem in many scientific applications. This paper introduces the concept of K-regression, which divides a random sample of size n into K clusters such that the observations within each cluster exhibit an identical linear pattern of dependence, and the observations in different clusters exhibit distinctive structures of linear dependence. We estimate the coefficients of the clustering regressions through minimizing the within cluster ℓ1 and ℓ2 loss functions. From the asymptotic perspective, the resulting estimates obtained with either the ℓ1 or the ℓ2 loss are strongly consistent and asymptotically normal. From the non-asymptotic perspective, we further explore the conditions under which the models are identifiable and the algorithms are convergent. Furthermore, we propose a tailored Bayesian Information Criterion (BIC) designed specifically for regression-based clustering. Through extensive simulations and an application to clinical trial subgroup analysis, we demonstrate the effectiveness of K-regression. Numerical results highlight that, in the presence of heterogeneity, ℓ1K-regression outperforms alternative methods (including ℓ2K-regression) in coefficient estimation, cluster number determination, and subgroup classification while maintaining computational efficiency. These advantages make ℓ1K-regression particularly appealing for large-scale data analysis, especially when heterogeneous subpopulations are present.