Multi-Agent Reinforcement Learning for User Scheduling in Coordinated Beamforming

Coordinated beamforming in multi-access-point (AP) systems has been proposed to address the high throughput and low latency demands of future wireless networks. To tackle the challenge of distributed fair user scheduling with reduced overhead, a multi-agent reinforcement learning (MARL)-based user scheduling scheme is introduced for a two-AP setup. This approach aims to determine user scheduling and power allocation at each AP distributively across consecutive time slots to enhance sum rates while ensuring fairness. In particular, it employs Multi-Agent Deep Deterministic Policy Gradient (MADDPG) with a multi-head self-attention mechanism to map channel information to AP behaviors. Additionally, a general CSI collection and information exchange scheme between the two APs is proposed to guarantee acquisition of necessary information. Simulation results illustrate the effectiveness of the proposed coordination framework, demonstrating learning convergence, and improvements of data rate optimization and fairness.