Deep-Reinforcement-Learning-Based AoI-Aware Resource Allocation for RIS-Aided IoV Networks

Deep-Reinforcement-Learning-Based AoI-Aware Resource Allocation for RIS-Aided IoV Networks

17 Jun 2024 | Kangwei Qi, Qiong Wu, Senior Member, IEEE, Pingyi Fan, Senior Member, IEEE, Nan Cheng, Senior Member, IEEE, Wen Chen, Senior Member, IEEE, Jiangzhou Wang, Fellow, IEEE and Khaled B. Letaief , Fellow, IEEE
This paper addresses the optimization of resource allocation and phase-shift control in RIS-assisted vehicular communication networks, focusing on minimizing the age of information (AoI) of V2I links and maximizing the payload transmission success rate of V2V links. The authors propose a deep reinforcement learning (DRL) approach, specifically the soft actor-critic (SAC) algorithm, to solve this problem. The system model includes a vehicular network with RIS-assisted V2I and V2V links, where vehicles are randomly distributed and interact with a base station (BS) and RIS. The paper formulates the optimization problem as a Markov decision process (MDP) and uses SAC to find the optimal policy. The SAC algorithm is chosen for its ability to handle high-dimensional state spaces and continuous actions, which are crucial in vehicular communication. The proposed method is compared with other algorithms such as PPO, TD3, and DDPG, and the results show that the SAC-based approach outperforms them in terms of convergence speed, cumulative reward, AoI performance, and payload transmission probability. The simulation results also demonstrate the effectiveness of the proposed scheme in various scenarios, including varying V2I transmission power, V2V payload size, and the number of RIS elements.This paper addresses the optimization of resource allocation and phase-shift control in RIS-assisted vehicular communication networks, focusing on minimizing the age of information (AoI) of V2I links and maximizing the payload transmission success rate of V2V links. The authors propose a deep reinforcement learning (DRL) approach, specifically the soft actor-critic (SAC) algorithm, to solve this problem. The system model includes a vehicular network with RIS-assisted V2I and V2V links, where vehicles are randomly distributed and interact with a base station (BS) and RIS. The paper formulates the optimization problem as a Markov decision process (MDP) and uses SAC to find the optimal policy. The SAC algorithm is chosen for its ability to handle high-dimensional state spaces and continuous actions, which are crucial in vehicular communication. The proposed method is compared with other algorithms such as PPO, TD3, and DDPG, and the results show that the SAC-based approach outperforms them in terms of convergence speed, cumulative reward, AoI performance, and payload transmission probability. The simulation results also demonstrate the effectiveness of the proposed scheme in various scenarios, including varying V2I transmission power, V2V payload size, and the number of RIS elements.
Reach us at info@study.space
[slides and audio] Deep-Reinforcement-Learning-Based AoI-Aware Resource Allocation for RIS-Aided IoV Networks