A power control algorithm based on Dyna-Q learning for ultra-dense networks

Xinyong Jia; Yi Wang; Jinquan Wang; Jiaqian Ding; Shixia Qiao

doi:10.1117/12.3004681

9 October 2023 A power control algorithm based on Dyna-Q learning for ultra-dense networks

Xinyong Jia, Yi Wang, Jinquan Wang, Jiaqian Ding, Shixia Qiao

Proceedings Volume 12791, Third International Conference on Advanced Algorithms and Neural Networks (AANN 2023); 127910A (2023) https://doi.org/10.1117/12.3004681
Event: Third International Conference on Advanced Algorithms and Neural Networks (AANN 2023), 2023, Qingdao, SD, China

Abstract

With ultra-dense networks, the large number of densely deployed low-power base stations creates more serious interference problems for the network. To address this problem, we introduce Dyna-Q learning into the power control problem of ultra-dense networks and propose a power control algorithm based on Dyna-Q learning. Firstly, we build an experience pool to store the state and selected actions of the agent during the operation; Secondly, the agent observes the current state, uses an action selection strategy to choose the next action, and then interacts with the environment to update the Q table; Finally, the agent learn again based on the virtual experience provided by the experience pool. The simulation results show that the normalized capacity of the method in this paper is improved by 0.88b/s/Hz on average and the resource consumption is reduced by 9.9% on average compared with the existing methods.

(2023) Published by SPIE. Downloading of the abstract is permitted for personal use only.

Citation Download Citation

Xinyong Jia, Yi Wang, Jinquan Wang, Jiaqian Ding, and Shixia Qiao "A power control algorithm based on Dyna-Q learning for ultra-dense networks", Proc. SPIE 12791, Third International Conference on Advanced Algorithms and Neural Networks (AANN 2023), 127910A (9 October 2023); https://doi.org/10.1117/12.3004681

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available