Optimizing Federated Learning with Heterogeneous Edge Devices

In the era of Internet of thing (IoT), data is massively generated by the distributed sensors. Federated learning (FL) has emerged as a privacy-preserving and secure DL framework to incorporate knowledge from data stored in different locations and train a DL model on the combined data without revealing the local data to a centralized server. This approach is limited by several challenges including the computation and communication cost. In this paper, we propose a novel FL framework that reduces the computation and communication cost through model pruning. It also, considers the device and data heterogeneity in IoT devices. This paper considers the problem of how to stochastically measure a large and complex information field (i.e., the criticality of parameters in local DL models based on local datasets) with optimally few observations. We adopt sparsity-promoting transform domain regularization and integrate compressive sensing with Bayesian learning to improve the measurement accuracy of criticality values in large-scale information fields while minimizing the overall computational efforts and time. The proposed approach is evaluated on two model architectures and two datasets. We demonstrate that our FL framework effectively addresses the heterogeneity present in local devices while surpassing the performance of state-of-the-art FL approaches, both with IID (independent and identically distributed) and non-IID (non-independent and identically distributed) data distributions.