How Global Observation Works in Federated Learning: Integrating Vertical Training Into Horizontal Federated Learning

Federated learning (FL) has recently emerged as an innovative paradigm to train models among distributed agents. Conventional FL considers the center as an aggregator and trains from distributed data, while the collected global information at the center is not effectively utilized. Thus, the restricted information from local observations may limit the model accuracy. If FL can introduce data sets from the network server, the distributed models may be largely improved by the extra global information. Since network agents may not be completely trusted, the center cannot directly broadcast its raw data for security concern. Then, how to combine the central sets with FL? In this article, we propose to add a learning model at the center, which obtains the central sets as input. The outputs can be transmitted to network agents and integrated into local models instead of the raw data. The central and local models could be trained to form an integration for intelligent inference. Then, what is the integrated performance gain comparing with the original horizontal FL (HFL) and how to implement it? To figure out these two problems, we propose the vertical-HFL (VHFL) scheme, where models of the center and agents are trained collaboratively. We further analyze its convergence and the related communication channel, proposing the theoretical bounds to guide the network implementation of VHFL. Some simulation results will demonstrate the effectiveness of our proposed VHFL scheme. It is expected that VHFL will be an important block for the next generation of smart services.