MARL Based Cooperative Passive Beamforming Design for Multi-IRS Aided Networks

Intelligent Reflecting Surface (IRS) is a planar surface that allows the signal transmission to be artificially manipulated. Communication aided by multiple IRSs is proven to be more effective in aspects such as transmission rate and signal coverage, but it raises complex coordination in issues such as channel estimation, resource allocation, and active/passive beamforming design problems for the Base Station (BS), IRSs and User Equipments (UEs). In this paper, we discuss the cooperative passive beamforming problem for multiple IRSs. First, we develop a multi-user communication scenario, in which multiple discrete phase shift IRSs are deployed to assist the transmission. Then, we formulate the cooperative passive beam-forming problem to maximize the sum transmission rate within a cellular multi-user network. To address this problem, we discuss a Reinforcement Learning (RL) approach, i.e., to transform the problem into a Multi-agent Markov Decision Process (MA MDP) and raise a Q-learning-based solution. Three different designs of the reward function are tested. Simulation results regarding the performance under different IRS and RL settings are given, which show that the proposed algorithm is capable of reaching a favorable passive beamforming solution in limited iterations and is effective in both convergence and long-term performance.