A Deep Gradient Compression Model Based on Tensor-Train Decomposition for Federated Learning

Federated Learning (FL) is an emerging artificial intelligence (AI) technology where the goal is to train a high-quality centralized model. However, one of challenges that FL faces is that it requires a large communication cost since it needs to exchange model parameters at each model update. In this paper, we propose a gradient compression algorithm with Tensor-Train (TT) decomposition to reduce the communication cost, where it uses low-di-mensional matrices or tensors to represent high-dimensional weight parameters. Simulation results show that our method can obtain a higher compression ratio while still achieving better accuracy compared with the other compression algorithms for FL.