Computer Science

Computer Science ›› 2025, Vol. 52 ›› Issue (3): 338-348.doi: 10.11896/jsjkx.240100091

• Computer Network • Previous Articles     Next Articles

Graph Reinforcement Learning Based Multi-edge Cooperative Load Balancing Method

ZHENG Longhai1,2,3, XIAO Bohuai1,2,3, YAO Zewei1,2,3, CHEN Xing1,2,3, MO Yuchang4   

  1. 1 College of Computer and Data Science,Fuzhou University,Fuzhou 350116,China
    2 Engineering Research Center of Big Data Intelligence,Ministry of Education,Fuzhou 350002,China
    3 Fujian Key Laboratory of Network Computing and Intelligent Information Processing,Fuzhou University,Fuzhou 350116,China
    4 Fujian Province University Key Laboratory of Computational Science,Huaqiao University,Quanzhou,Fujian 362021,China
  • Received:2024-01-08 Revised:2024-05-31 Online:2025-03-15 Published:2025-03-07
  • About author:ZHENG Longhai,born in 1999,postgraduate.His main research interests include load balancing and task offloa-ding.
    CHEN Xing,born in 1985,Ph.D,professor,Ph.D supervisor,is a senior member of CCF(No.35725M).His main research interests include software engineering,system software and cloud computing.
  • Supported by:
    National Natural Science Foundation of China(62072108),Fujian Province Technology and Economy Integration Service Platform(2023XRH001) and Fuzhou-Xiamen-Quanzhou National Independent Innovation Demonstration Zone Collaborative Innovation Platform(2022FX5).

PDF (PC)

160

Abstract: In mobile edge computing,devices can effectively relieve latency and energy consumption by offloading computation-intensive tasks to nearby edge servers.In order to improve the quality of service,edge servers need to collaborate with each other rather than working alone.For the load balancing problem of multi-edge collaboration,the existing solutions often depend on accurate mathematical models or make fair use of edge topological relationships.To solve this problem,an offloading decision-ma-king method based on graph reinforcement learning is proposed in this paper.Firstly,the load balancing scenario with multi-edge collaboration is abstracted as graph data,then a graph embedding process based on graph convolutional neural network is used to extract the information features of the graph,for assisting the deep Q-network to make offloading decisions,and finally the objective load balancing plan is found through a centralized feedback-control mechanism.Simulation experiments are conducted in multiple scenarios,the results verify the effectiveness of the proposed method in shortening the average response latency of the tasks,and the load balancing effect which is better than the comparison algorithms and close to the ideal plan can be obtained in a short period of time.

Key words: Multi-edge collaboration, Load balancing, Task offloading, Graph neural network, Deep reinforcement learning

CLC Number: 

  • TP338
[1]YAO Z,LIN J,HU J,et al.PSO-GA based approach to multi-edge load balancing[J].Computer Science,2021,48(S2):456-463.
[2]XU J,CHEN L,REN S.Online learning for offloading and autoscaling in energy harvesting mobile edge computing[J].IEEE Transactions on Cognitive Communications and Networking,2017,3(3):361-373.
[3]ALASMARI K R,GREEN R C,ALAM M.Mobile edge offloa-ding using markov decision processes[C]//Edge Computing-EDGE 2018:Second International Conference,Held as Part of the Services Conference Federation,SCF 2018,Seattle,WA,USA,June 25－30,2018,Proceedings 2.Springer International Publishing,2018:80-90.
[4]SUN Z,MO Y,YU C.Graph reinforcement learning based taskoffloading for multi-access edge computing[J].IEEE Internet of Things Journal,2023,10(4):3138-3150.
[5]CABRERA A,ACOSTA A,ALMEIDA F,et al.A dynamicmulti-objective approach for dynamic load balancing in heterogeneous systems[J].IEEE Transactions on Parallel and Distributed Systems,2020,31(10):2421-2434.
[6]MEHRABI A,SIEKKINEN M,YLÄ-JÄÄSKI A.Edge computing assisted adaptive mobile video streaming[J].IEEE Transactions on Mobile Computing,2018,18(4):787-800.
[7]WANG Z,ZHANG W,JIN X,et al.An optimal edge serverplacement approach for cost reduction and load balancing in intelligent manufacturing[J].The Journal of Supercomputing,2022,78(3):4032-4056.
[8]YANG L,YAO H,WANG J,et al.Multi-UAV-enabled load-balance mobile-edge computing for IoT networks[J].IEEE Internet of Things Journal,2020,7(8):6898-6908.
[9]JIA M,LIANG W,XU Z,et al.Qos-aware cloudlet load balancing in wireless metropolitan area networks[J].IEEE Transactions on Cloud Computing,2018,8(2):623-634.
[10]TRAN T X,POMPILI D.Joint task offloading and resource allocation for multi-server mobile-edge computing networks[J].IEEE Transactions on Vehicular Technology,2018,68(1):856-868.
[11]XU X,LIU K,DAI P,et al.Joint task offloading and resource optimization in noma-based vehicular edge computing:A game-theoretic drl approach[J].Journal of Systems Architecture,2023,134:102780.
[12]LIU Q,XIA T,CHENG L,et al.Deep reinforcement learning for load-balancing aware network control in IoT edge systems[J].IEEE Transactions on Parallel and Distributed Systems,2021,33(6):1491-1502.
[13]LI J,LUO G,CHENG N,et al.An end-to-end load balancer based on deep learning for vehicular network traffic control[J].IEEE Internet of Things Journal,2018,6(1):953-966.
[14]XU Y,XU W,WANG Z,et al.Load balancing for ultradensenetworks:A deep reinforcement learning-based approach[J].IEEE Internet of Things Journal,2019,6(6):9399-9412.
[15]CHEN X,HU J,CHEN Z,et al.A reinforcement learning-empowered feedback control system for industrial internet of things[J].IEEE Transactions on Industrial Informatics,2021,18(4):2724-2733.
[16]LI J,GAO H,LV T,et al.Deep reinforcement learning basedcomputation offloading and resource allocation for MEC[C]//2018 IEEE Wireless Communications and Networking Confe-rence(WCNC).IEEE,2018:1-6.
[17]ZHANG S,GU H,CHI K,et al.Drl-based partial offloading for maximizing sum computation rate of wireless powered mobile edge computing network[J].IEEE Transactions on Wireless Communications,2022,21(12):10934-10948.
[18]LI K,NI W,YUAN X,et al.Deep-Graph-Based Reinforcement Learning for Joint Cruise Control and Task Offloading for Aerial Edge Internet of Things(EdgeIoT)[J].IEEE Internet of Things Journal,2022,9(21):21676-21686.
[19]CHEN J,YANG Y,WANG C,et al.Multitask offloading strategy optimization based on directed acyclic graphs for edge computing[J].IEEE Internet of Things Journal,2021,9(12):9367-9378.
[20]DAI A,LI R,ZHAO Z,et al.Graph convolutional multi-agentreinforcement learning for UAV coverage control[C]//2020 International Conference on Wireless Communications and Signal Processing(WCSP).IEEE,2020:1106-1111.
[21]ZHANG J,DU J,SHEN Y,et al.Dynamic computation offloa-ding with energy harvesting devices:A hybrid-decision-based deep reinforcement learning approach[J].IEEE Internet of Things Journal,2020,7(10):9303-9317.
[22]GAO Z,YANG L,DAI Y.Fast Adaptive Task Offloading and Resource Allocation in Large-Scale MEC Systems via Multi-Agent Graph Reinforcement Learning[J].IEEE Internet of Things Journal,2023,11(1):758-776.
[23]CAO X,TANG G,GUO D,et al.Edge federation:Towards an integrated service provisioning model[J].IEEE/ACM Transactions on Networking,2020,28(3):1116-1129.
[24]MNIH V,KAVUKCUOGLU K,SILVER D,et al.Human-level control through deep reinforcement learning[J].Nature,2015,518(7540):529-533.
[25]WANG S,ZHAO Y,XU J,et al.Edge server placement in mobile edge computing[J].Journal of Parallel and Distributed Computing,2019,127:160-168.
[26]WANG S,ZHAO Y,HUANG L,et al.QoS prediction for ser-vice recommendations in mobile edge computing[J].Journal of Parallel and Distributed Computing,2019,127:134-144.
[27]GUO Y,WANG S,ZHOU A,et al.User allocation-aware edge cloud placement in mobile edge computing[J].Software:Practice and Experience,2020,50(5):489-502.
[28]WANG S,GUO Y,ZHANG N,et al.Delay-aware microservice coordination in mobile edge computing:A reinforcement learning approach[J].IEEE Transactions on Mobile Computing,2019,20(3):939-951.
[29]VILAPLANA J,SOLSONA F,TEIXIDÓ I,et al.A queuingtheory model for cloud computing[J].The Journal of Supercomputing,2014,69:492-507.
[30]CHEN X,YAO Z,CHEN Z,et al.Load Balancing for Multi-Edge Collaboration in Wireless Metropolitan Area Networks:A Two-Stage Decision-Making Approach[J].IEEE Internet of Things Journal,2023,10(19):17124-17136.
[31]WU Y,GUO K,HUANG J,et al.Secrecy-based energy-efficient data offloading via dual connectivity over unlicensed spectrums[J].IEEE Journal on Selected Areas in Communications,2016,34(12):3252-3270.
[1] YANG Yingxiu, CHEN Hongmei, ZHOU Lihua , XIAO Qing. Heterogeneous Graph Attention Network Based on Data Augmentation [J]. Computer Science, 2025, 52(3): 180-187.
[2] LI Shao, JIANG Fangting, YANG Xinyan, LIANG Gang. Rumor Detection on Potential Hot Topics with Bi-directional Graph Attention Network [J]. Computer Science, 2025, 52(3): 277-286.
[3] HU Haifeng, ZHU Yiwen, ZHAO Haitao. Network Slicing End-to-end Latency Prediction Based on Heterogeneous Graph Neural Network [J]. Computer Science, 2025, 52(3): 349-358.
[4] DU Likuan, LIU Chen, WANG Junlu, SONG Baoyan. Self-learning Star Chain Space Adaptive Allocation Method [J]. Computer Science, 2025, 52(3): 359-365.
[5] HUO Xingpeng, SHA Letian, LIU Jianwen, WU Shang, SU Ziyue. Windows Domain Penetration Testing Attack Path Generation Based on Deep Reinforcement Learning [J]. Computer Science, 2025, 52(3): 400-406.
[6] YUAN Ye, CHEN Ming, WU Anbiao, WANG Yishu. Graph Anomaly Detection Model Based on Personalized PageRank and Contrastive Learning [J]. Computer Science, 2025, 52(2): 80-90.
[7] ZHENG Wenping, HAN Yiheng, LIU Meilin. Node Classification Algorithm Fusing High-order Group Structure Information [J]. Computer Science, 2025, 52(2): 107-115.
[8] WANG Yijie, GAO Guoju, SUN Yu'e, HUANG He. Flow Cardinality Estimation Method Based on Distributed Sketch in SDN [J]. Computer Science, 2025, 52(2): 268-278.
[9] XU Donghong, LI Bin, QI Yong. Task Scheduling Strategy Based on Improved A2C Algorithm for Cloud Data Center [J]. Computer Science, 2025, 52(2): 310-322.
[10] WANG Xin, XIONG Shubo, SUN Lingyun. Federated Graph Learning:Problems,Methods and Challenges [J]. Computer Science, 2025, 52(1): 362-373.
[11] CHEN Liang, SUN Cong. Deep-learning Based DKOM Attack Detection for Linux System [J]. Computer Science, 2024, 51(9): 383-392.
[12] WANG Tianjiu, LIU Quan, WU Lan. Offline Reinforcement Learning Algorithm for Conservative Q-learning Based on Uncertainty Weight [J]. Computer Science, 2024, 51(9): 265-272.
[13] ZHOU Wenhui, PENG Qinghua, XIE Lei. Study on Adaptive Cloud-Edge Collaborative Scheduling Methods for Multi-object State Perception [J]. Computer Science, 2024, 51(9): 319-330.
[14] TANG Ying, WANG Baohui. Study on SSL/TLS Encrypted Malicious Traffic Detection Algorithm Based on Graph Neural Networks [J]. Computer Science, 2024, 51(9): 365-370.
[15] CHEN Shanshan, YAO Subin. Study on Recommendation Algorithms Based on Knowledge Graph and Neighbor PerceptionAttention Mechanism [J]. Computer Science, 2024, 51(8): 313-323.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.