Heterogeneous federated learning with client collaboration graphs

Federated learning is a distributed machine learning paradigm for protecting data privacy. Clients participating in federated learning usually have different hardware resources and training data, but traditional federated learning methods often ignore the multi-level collaborative relationships among heterogeneous clients in terms of data and resources, which leads to insufficient collaboration and limited convergence performance. To address this problem, this paper proposes a heterogeneous federated learning method based on client collaboration graph (HFLCG), which aims to mine the multi-level collaboration relationships among heterogeneous clients and realize efficient and fine-grained client collaboration. In the HFLCG method, clients are categorized into multiple clusters by introducing quantitative metrics of client computing power. According to the resource differences between clusters, a directed asymmetric topology is constructed for the resource collaboration relationship graph, so that the high-capacity clusters accelerate the convergence of low-capacity clusters. At the same time, the data knowledge of the low-capacity clusters complements that of the high-capacity clusters, which effectively alleviates the problems of slow convergence of the low-capacity clients and the misalignment of the knowledge exchanges in the traditional federated learning. In addition, by constructing a weight-adaptive data collaboration graph within the client clusters, it can better cope with different levels of data heterogeneity. The two-level collaboration graphs are interlinked so that clients can fully absorb the data knowledge while being assisted by high-resource clients as much as possible to solve the data and resource heterogeneity problems simultaneously. Experiments show that the proposed method takes at least 47.34% less time than the baseline method to achieve the same target accuracy, and has better robustness under different resources and data heterogeneity.