Efficient Model Compression via Global Sparsification for Over-the-Air Federated Learning

Federated learning (FL) is an attractive distributed intelligence paradigm renowned for its privacy-preserving advan-tages. However, the extensive communication overhead caused by intermediate model exchange is an inevitable challenge faced by wireless FL. In this work, we investigate efficient sparsification schemes specifically tailored to FL systems based on over- the-air computation (OAC). In particular, considering the distinctive transmission characteristics of OAC-based FL, we propose the Top-Rand global sparsification scheme that combines the advantages of the conventional Top-k and Rand-k sparsification techniques. Additionally, we observe that sparsifying the shallow layers of FL models typically results in greater accuracy degradation and accordingly propose a layer-wise sparsification algorithm. Simulation results reveal that the proposed Top-Rand sparsification can improve efficiency more effectively than the existing methods at diverse client scales, and that the layer-wise design can bring further improvements.