Multi-Agent Reinforcement Learning-Based Joint Precoding and Phase Shift Optimization for RIS-Aided Cell-Free Massive MIMO Systems

Cell-free (CF) massive multiple-input multiple-output (mMIMO) is a promising technique for achieving high spectral efficiency (SE) using multiple distributed access points (APs). However, harsh propagation environments often lead to significant communication performance degradation due to high penetration loss. To overcome this issue, we introduce the reconfigurable intelligent surface (RIS) into the CF mMIMO system as a low-cost and power-efficient solution. In this paper, we focus on optimizing the joint precoding design of the RIS-aided CF mMIMO system to maximize the sum SE. This involves optimizing the precoding matrix at the APs and the reflection coefficients at the RIS. To tackle this problem, we propose a fully distributed multi-agent reinforcement learning (MARL) algorithm that incorporates fuzzy logic (FL). Unlike conventional approaches that rely on alternating optimization techniques, our FL-based MARL algorithm only requires local channel state information, which reduces the need for high backhaul capacity. Simulation results demonstrate that our proposed FL-MARL algorithm effectively reduces computational complexity while achieving similar performance as conventional MARL methods.

I. Introduction

The sixth-generation (6G) network will be a vital component in all parts of future society, industry, and life, given its primary mission to fulfill the communication needs of humans and intelligent machines [1]. The integration of distributed networks and massive MIMO confers notable advantages upon an ultra-dense network known as cell-free (CF) massive multiple-input multiple-output (mMIMO). In the context of CF mMIMO networks, a substantial array of distributed access points (APs) collectively cater to a limited user base using concurrent time-frequency resources, while all base stations (BSs) are linked to a central processing unit (CPU) through backhaul wireless connections [2], [3].