Joint Sparsification and Quantization for Wireless Federated Learning under Communication Constraints

Communication efficiency is one of the key bottlenecks in Federated Learning (FL). Compression techniques, such as sparsification and quantization, are used to reduce communication overhead. However, joint designs of these techniques under communication constraints are not well-explored. This paper investigates the joint uplink compression problem in communication-constrained FL systems. We propose a Block-TopK sparsification scheme to reduce the proportion of bits used for locating entries of a sparsified vector. Considering the communication constraints, an optimization formulation is proposed to minimize the compression error. By solving the optimization problem, our joint compression method provides a better trade-off between sparsity budget and bit width. Numerical results demonstrate that our approach achieves 99.96% of baseline accuracy with only 1.56% of the baseline communication overhead when training ResNet-18 on CIFAR-10.