Computer Science

Computer Science ›› 2021, Vol. 48 ›› Issue (11): 363-371.doi: 10.11896/jsjkx.201000008

• Computer Network • Previous Articles     Next Articles

Reinforcement Learning Based Dynamic Basestation Orchestration for High Energy Efficiency

ZENG De-ze1, LI Yue-peng1, ZHAO Yu-yang1, GU Lin2   

  1. 1 School of Computer Science,China University of Geosciences,Wuhan 430074,China
    2 School of Computer Science and Technology,Huazhong University of Science and Technology,Wuhan 430074,China
  • Received:2020-10-03 Revised:2021-04-10 Online:2021-11-15 Published:2021-11-10
  • About author:ZENG De-ze,born in 1984,Ph.D,professor,Ph.D supervisor,is a member of China Computer Federation.His main research interests include edge computing and artificial intelligence.
    GU Lin,born in 1985,Ph.D,associate professor,is a member of China Computer Federation.Her main research interests include edge computing and artificial intelligence.
  • Supported by:
    National Natural Science Foundation of China(61772480,61972171,62073300) and Open Research Projects of Zhijiang Lab(2021KE0AB02).

PDF (PC)

906

Abstract: The mutual promotion of mobile communication technology and mobile communication industry has achieved unprecedented prosperity in the mobile Internet era.The explosion of mobile devices,expansion of the network scale,improvement of service requirements are driving the next technological revolution in wireless networks.5G meets the requirements for the thousand-fold improvement of service performance through intensive network deployment,but co-channel interference and bursty request problems make the energy consumption of this solution very huge.In order to support 5G network to provide energy-efficient and high-performance services,it is imperative to upgrade and improve the management scheme of mobile networks.In this article,we use a short-cycle management framework with cache queues to achieve agile and smooth management of request burst scenarios to avoid dramatic fluctuations in service quality due to request bursts.We use deep reinforcement learning to learn the user distribution and communication needs,and infer the load change rules of the base station,and then realize the pre-scheduling and pre-allocation of energy,while ensuring the quality of service and improving the energy efficiency.Compared with the classic DQN algorithm,the two-buffer DQN algorithm proposed in this paper can provide nearly 20% acceleration in convergence.In terms of decision performance,it can save 4.8% energy consumption compared to the currently widely used keep on strategy.

Key words: Base station sleep, Deep reinforcement learning, Double buffer, Heterogeneous network, Mobile network management, Request burst

CLC Number: 

  • TP391
[1]OH E,KRISHNAMACHARI B,LIU X,et al.Toward dynamic energy efficient operation of cellular network infrastructure[J].IEEE Communications Magazine,2011,49(6):56-61.
[2]LÄHDEKORPI P,HRONEC M,JOLMA P,et al.Energy efficiency of 5G mobile networks with base station sleep modes[C]//2017 IEEE Conference on Standards for Communications and Networking (CSCN).IEEE,2017:163-168.
[3]ONIRETI O,MOHAMED A,PERVAIZ H,et al.Analytical approach to base station sleep mode power consumption and sleep depth[C]//2017 IEEE 28th Annual International Symposium on Personal,Indoor,and Mobile Radio Communications (PIMRC).IEEE,2017:1-7.
[4]ONIRETI O,MOHAMED A,PERVAIZ H,et al.A tractable ap-proach to base station sleep mode power consumption and deactivation latency[C]//2018 IEEE 29th Annual International Symposium on Personal,Indoor and Mobile Radio Communications (PIMRC).IEEE,2018:123-128.
[5]PERVAIZ H,ONIRETI O,MOHAMED A,et al.Energy-efficient and load-proportional eNodeB for 5G user-centric networks:a multilevel sleep strategy mechanism[J].IEEE Vehicular Technology Magazine,2018,13(4):51-59.
[6]LI J,WANG H,WANG X,et al.Optimized sleep strategy based on clustering in dense heterogeneous networks[J].EURASIP Journal on Wireless Communications and Networking,2018,2018(1):1-10.
[7]RATHEESH R,VETRIVELAN P.Energy efficiency based onrelay station deployment and sleep mode activation of eNBs for 4G LTE-A network[J].Automatika,2019,60(3):322-331.
[8]KLAPEZ M,GRAZIA C A,CASONI M.Energy Savings ofSleep Modes Enabled by 5G Software-Defined Heterogeneous Networks[C]//2018 IEEE 4th International Forum on Research and Technology for Society and Industry (RTSI).IEEE,2018:1-6.
[9]JAWAD A M,JAWAD H M,NORDIN R,et al.Wireless power transfer with magnetic resonator coupling and sleep/active strategy for a drone charging station in smart agriculture[J].IEEE Access,2019,7:139839-139851.
[10]WU J,BAO Y,MIAO G,et al.Base station sleeping and power control for bursty traffic in cellular networks[C]//2014 IEEE International Conference on Communications Workshops (ICC).IEEE,2014:837-841.
[11]WU J,BAO Y,MIAO G,et al.Base-station sleeping control and power matching for energy-delay tradeoffs with bursty traffic[J].IEEE Transactions on Vehicular Technology,2015,65(5):3657-3675.
[12]LIU J,KRISHNAMACHARI B,ZHOU S,et al.Deepnap:Data-driven base station sleeping operations through deep reinforcement learning[J].IEEE Internet of Things Journal,2018,5(6):4273-4282.
[13]GHADIMI E,CALABRESE F D,PETERS G,et al.A reinforcement learning approach to power control and rate adaptation in cellular networks[C]//2017 IEEE International Conference on Communications (ICC).IEEE,2017:1-7.
[14]WANG L,PETERS G,LIANG Y C,et al.Intelligent User-Centric Networks:Learning-Based Downlink CoMP Region Brea-thing[J].IEEE Transactions on Vehicular Technology,2020,69(5):5583-5597.
[15]LIU Q,SHI J.Base station sleep and spectrum allocation in he-terogeneous ultra-dense networks[J].Wireless Personal Communications,2018,98(4):3611-3627.
[16]NIU Z,GUO X,ZHOU S,et al.Characterizing energy-delaytradeoff in hyper-cellular networks with base station sleeping control[J].IEEE Journal on Selected Areas in Communications,2015,33(4):641-650.
[17]CHEN X,WU J,CAI Y,et al.Energy-efficiency oriented traffic offloading in wireless networks:a brief survey and a learning approach for heterogeneous cellular networks[J].IEEE Journal on Selected Areas in Communications,2015,33(4):627-640.
[18]FENG M,MAO S,JIANG T.Boost:Base station on off swi-tching strategy for energy efficient massive mimo hetnets[C]// IEEE INFOCOM 2016 The 35th Annual IEEE International Conference on Computer Communications.IEEE,2016:1-9.
[19]SAMARAKOON S,BENNIS M,SAAD W,et al.Opportunistic sleep mode strategies in wireless small cell networks[C]//2014 IEEE International Conference on Communications(ICC).IEEE,2014:2707-2712.
[20]SALEM F E,ALTMAN Z,GATI A,et al.Reinforcement lear-ning approach for advanced sleep modes management in 5G networks[C]//2018 IEEE 88th Vehicular Technology Conference (VTC-Fall).IEEE,2018:1-5.
[21]EL-AMINE A,ITURRALDE M,HASSAN H A H,et al.A distributed Q-Learning approach for adaptive sleep modes in 5G networks[C]//2019 IEEE Wireless Communications and Networking Conference (WCNC).IEEE,2019:1-6.
[22]ZHANG Y T.A Deep Reinforcement Learning based Dynamic C-RAN Resource Allocation Method[J].Journal of Chinese Computer Systems,2021,42(1):132-136.
[1] HUANG Li, ZHU Yan, LI Chun-ping. Author’s Academic Behavior Prediction Based on Heterogeneous Network Representation Learning [J]. Computer Science, 2022, 49(9): 76-82.
[2] YU Bin, LI Xue-hua, PAN Chun-yu, LI Na. Edge-Cloud Collaborative Resource Allocation Algorithm Based on Deep Reinforcement Learning [J]. Computer Science, 2022, 49(7): 248-253.
[3] LI Meng-fei, MAO Ying-chi, TU Zi-jian, WANG Xuan, XU Shu-fang. Server-reliability Task Offloading Strategy Based on Deep Deterministic Policy Gradient [J]. Computer Science, 2022, 49(7): 271-279.
[4] XIE Wan-cheng, LI Bin, DAI Yue-yue. PPO Based Task Offloading Scheme in Aerial Reconfigurable Intelligent Surface-assisted Edge Computing [J]. Computer Science, 2022, 49(6): 3-11.
[5] HONG Zhi-li, LAI Jun, CAO Lei, CHEN Xi-liang, XU Zhi-xiong. Study on Intelligent Recommendation Method of Dueling Network Reinforcement Learning Based on Regret Exploration [J]. Computer Science, 2022, 49(6): 149-157.
[6] LI Peng, YI Xiu-wen, QI De-kang, DUAN Zhe-wen, LI Tian-rui. Heating Strategy Optimization Method Based on Deep Learning [J]. Computer Science, 2022, 49(4): 263-268.
[7] OUYANG Zhuo, ZHOU Si-yuan, LYU Yong, TAN Guo-ping, ZHANG Yue, XIANG Liang-liang. DRL-based Vehicle Control Strategy for Signal-free Intersections [J]. Computer Science, 2022, 49(3): 46-51.
[8] PU Shi, ZHAO Wei-dong. Community Detection Algorithm for Dynamic Academic Network [J]. Computer Science, 2022, 49(1): 89-94.
[9] DAI Shan-shan, LIU Quan. Action Constrained Deep Reinforcement Learning Based Safe Automatic Driving Method [J]. Computer Science, 2021, 48(9): 235-243.
[10] CHENG Zhao-wei, SHEN Hang, WANG Yue, WANG Min, BAI Guang-wei. Deep Reinforcement Learning Based UAV Assisted SVC Video Multicast [J]. Computer Science, 2021, 48(9): 271-277.
[11] ZHOU Shi-cheng, LIU Jing-ju, ZHONG Xiao-feng, LU Can-ju. Intelligent Penetration Testing Path Discovery Based on Deep Reinforcement Learning [J]. Computer Science, 2021, 48(7): 40-46.
[12] LI Bei-bei, SONG Jia-rui, DU Qing-yun, HE Jun-jiang. DRL-IDS:Deep Reinforcement Learning Based Intrusion Detection System for Industrial Internet of Things [J]. Computer Science, 2021, 48(7): 47-54.
[13] LIANG Jun-bin, ZHANG Hai-han, JIANG Chan, WANG Tian-shu. Research Progress of Task Offloading Based on Deep Reinforcement Learning in Mobile Edge Computing [J]. Computer Science, 2021, 48(7): 316-323.
[14] WANG Ying-kai, WANG Qing-shan. Reinforcement Learning Based Energy Allocation Strategy for Multi-access Wireless Communications with Energy Harvesting [J]. Computer Science, 2021, 48(7): 333-339.
[15] FAN Jia-kuan, WANG Hao-yue, ZHAO Sheng-yu, ZHOU Tian-yi, WANG Wei. Data-driven Methods for Quantitative Assessment and Enhancement of Open Source Contributions [J]. Computer Science, 2021, 48(5): 45-50.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.