Local Momentum Accelerated Based Non-IID Federated Learning Method

Federated Learning (FL), a distributed machine learning technology, has achieved significant results in privacy protection. However, in practical applications, client drift phenomena occur because of the Non-Independent and Identically Distributed (Non-IID) nature of data sources, leading to slow model convergence and performance degradation. To address this issue, this study proposes a Federated Local Momentum accelerated learning (FedLM) algorithm combined with the attention mechanism. FedLM introduces a global momentum term into local model updates, utilizing the global gradient information from previous rounds to smooth the current update process and correct the divergence of parameter update directions among heterogeneous clients, thereby reducing gradient oscillations and alleviating data heterogeneity issues. The attention mechanism dynamically adjusts the weight of each client in the global model update to improve the quality of the aggregation model. Experimental results show that FedLM achieves significantly better accuracy and stability than existing federated learning algorithms such as SCAFFOLD, FedCM, and Moon in image classification tasks with different levels of data heterogeneity, model structures, and datasets.