A Novel Incentive Mechanism for Federated Learning Over Wireless Communications

This article studies a federated learning system over wireless communications, where a parameter server shares a global model trained by distributed devices. Due to limited communication resources, not all devices can participate in the training process. To encourage suitable devices to participate, this article proposes a novel incentive mechanism, where the parameter server assigns rewards to the devices, and the devices make participation decisions to maximize their overall profit based on the obtained rewards and their energy costs. Based on the interaction between the parameter server and the devices, the proposed incentive mechanism is formulated as a bilevel optimization problem (BOP), in which the upper level optimizes reward factors for the parameter server and the lower level makes participation decisions for the devices. Note that each device needs to make an independent participation decision due to limited communication resources and privacy concerns. To solve this BOP, a bilevel optimization approach called BIMFL is proposed. BIMFL adopts multiagent reinforcement learning (MARL) to make independent participation decisions with local information at the lower level, and introduces multiagent meta-reinforcement learning to accelerate the training by incorporating meta-learning into MARL. Moreover, BIMFL utilizes covariance matrix adaptation evolutionary strategy to optimize reward factors at the upper level. The effectiveness of BIMFL is demonstrated on different datasets using multilayer perceptron and convolutional neural networks.

I. Introduction

Machine learning is a subfield of artificial intelligence, which trains models based on sample data to make intelligent predictions [1], [2]. In many machine learning applications, such as facial recognition [3] and medical diagnosis [4], [5], data is often collected through distributed devices. However, these devices may be hesitant to share data due to privacy concerns [6]. To address this challenge, federated learning (FL) has emerged as a distributed machine learning approach. FL enables distributed devices to collaboratively train a global model shared by the parameter server while keeping their collected privacy data locally [7], [8], [9], [10]. Compared with traditional machine learning methods, FL has several advantages, including privacy preservation and efficient resource utilization [11], [12].