FNCS: Federated Learning Strategy Based on Cosine Similarity under Resource Constraints

Federated learning has been widely applied in healthcare services and real-time object tracking. However, limited by communication resources, such as the server bandwidth, and the impact of client data heterogeneity, the convergence rates and accuracy of federated learning significantly drop. Hence, this study proposes a novel federated normalization learning strategy based on cosine similarity (FNCS). Starting from a new perspective of the relationship between local and global updates of the model, FNCS selects valuable clients to upload updates using cosine similarity. The regularization term is then inserted in the last layer of clients by utilizing cosine distance-based update divergence. Numerous experiments are conducted in PyTorch for accelerated validation. Results show that the high accuracy experiments are conducted on the complex CelebA dataset, and the communication rounds of FNCS are improved by 44.71% and 41.98% compared with FedAvg and FedProx, respectively.

I. Introduction

Artificial intelligence-based services pose data privacy, communication, and computational challenges that warrant prevailing concerns. As a distributed machine learning architecture, the proposal of federated learning (FL) [1]–[3] provides a new idea to address the above challenges. However, studies, such as those of the optimization of FL in real networks [4], are at an exploratory stage. At present, the federated averaging (FedAvg) algorithm proposed in the paper [5] assumes that all client data are random samples of real data distribution. Although the FedAvg algorithm obtains excellent training results in this case, the heterogeneous computing resources and different communication capabilities of clients may invalidate this assumption. Therefore, improving training efficiency for the successful deployment of FL in edge and mobile devices under resource-constrained and client heterogeneous environments is crucial [6], [7].