Computer Science

Computer Science ›› 2025, Vol. 52 ›› Issue (3): 338-348.doi: 10.11896/jsjkx.240100091

• Computer Network • Previous Articles     Next Articles

Graph Reinforcement Learning Based Multi-edge Cooperative Load Balancing Method

ZHENG Longhai1,2,3, XIAO Bohuai1,2,3, YAO Zewei1,2,3, CHEN Xing1,2,3, MO Yuchang4   

  1. 1 College of Computer and Data Science,Fuzhou University,Fuzhou 350116,China
    2 Engineering Research Center of Big Data Intelligence,Ministry of Education,Fuzhou 350002,China
    3 Fujian Key Laboratory of Network Computing and Intelligent Information Processing,Fuzhou University,Fuzhou 350116,China
    4 Fujian Province University Key Laboratory of Computational Science,Huaqiao University,Quanzhou,Fujian 362021,China
  • Received:2024-01-08 Revised:2024-05-31 Online:2025-03-15 Published:2025-03-07
  • About author:ZHENG Longhai,born in 1999,postgraduate.His main research interests include load balancing and task offloa-ding.
    CHEN Xing,born in 1985,Ph.D,professor,Ph.D supervisor,is a senior member of CCF(No.35725M).His main research interests include software engineering,system software and cloud computing.
  • Supported by:
    National Natural Science Foundation of China(62072108),Fujian Province Technology and Economy Integration Service Platform(2023XRH001) and Fuzhou-Xiamen-Quanzhou National Independent Innovation Demonstration Zone Collaborative Innovation Platform(2022FX5).

PDF (PC)

251

Abstract: In mobile edge computing,devices can effectively relieve latency and energy consumption by offloading computation-intensive tasks to nearby edge servers.In order to improve the quality of service,edge servers need to collaborate with each other rather than working alone.For the load balancing problem of multi-edge collaboration,the existing solutions often depend on accurate mathematical models or make fair use of edge topological relationships.To solve this problem,an offloading decision-ma-king method based on graph reinforcement learning is proposed in this paper.Firstly,the load balancing scenario with multi-edge collaboration is abstracted as graph data,then a graph embedding process based on graph convolutional neural network is used to extract the information features of the graph,for assisting the deep Q-network to make offloading decisions,and finally the objective load balancing plan is found through a centralized feedback-control mechanism.Simulation experiments are conducted in multiple scenarios,the results verify the effectiveness of the proposed method in shortening the average response latency of the tasks,and the load balancing effect which is better than the comparison algorithms and close to the ideal plan can be obtained in a short period of time.

Key words: Multi-edge collaboration, Load balancing, Task offloading, Graph neural network, Deep reinforcement learning

CLC Number: 

  • TP338
[1]YAO Z,LIN J,HU J,et al.PSO-GA based approach to multi-edge load balancing[J].Computer Science,2021,48(S2):456-463.
[2]XU J,CHEN L,REN S.Online learning for offloading and autoscaling in energy harvesting mobile edge computing[J].IEEE Transactions on Cognitive Communications and Networking,2017,3(3):361-373.
[3]ALASMARI K R,GREEN R C,ALAM M.Mobile edge offloa-ding using markov decision processes[C]//Edge Computing-EDGE 2018:Second International Conference,Held as Part of the Services Conference Federation,SCF 2018,Seattle,WA,USA,June 25－30,2018,Proceedings 2.Springer International Publishing,2018:80-90.
[4]SUN Z,MO Y,YU C.Graph reinforcement learning based taskoffloading for multi-access edge computing[J].IEEE Internet of Things Journal,2023,10(4):3138-3150.
[5]CABRERA A,ACOSTA A,ALMEIDA F,et al.A dynamicmulti-objective approach for dynamic load balancing in heterogeneous systems[J].IEEE Transactions on Parallel and Distributed Systems,2020,31(10):2421-2434.
[6]MEHRABI A,SIEKKINEN M,YLÄ-JÄÄSKI A.Edge computing assisted adaptive mobile video streaming[J].IEEE Transactions on Mobile Computing,2018,18(4):787-800.
[7]WANG Z,ZHANG W,JIN X,et al.An optimal edge serverplacement approach for cost reduction and load balancing in intelligent manufacturing[J].The Journal of Supercomputing,2022,78(3):4032-4056.
[8]YANG L,YAO H,WANG J,et al.Multi-UAV-enabled load-balance mobile-edge computing for IoT networks[J].IEEE Internet of Things Journal,2020,7(8):6898-6908.
[9]JIA M,LIANG W,XU Z,et al.Qos-aware cloudlet load balancing in wireless metropolitan area networks[J].IEEE Transactions on Cloud Computing,2018,8(2):623-634.
[10]TRAN T X,POMPILI D.Joint task offloading and resource allocation for multi-server mobile-edge computing networks[J].IEEE Transactions on Vehicular Technology,2018,68(1):856-868.
[11]XU X,LIU K,DAI P,et al.Joint task offloading and resource optimization in noma-based vehicular edge computing:A game-theoretic drl approach[J].Journal of Systems Architecture,2023,134:102780.
[12]LIU Q,XIA T,CHENG L,et al.Deep reinforcement learning for load-balancing aware network control in IoT edge systems[J].IEEE Transactions on Parallel and Distributed Systems,2021,33(6):1491-1502.
[13]LI J,LUO G,CHENG N,et al.An end-to-end load balancer based on deep learning for vehicular network traffic control[J].IEEE Internet of Things Journal,2018,6(1):953-966.
[14]XU Y,XU W,WANG Z,et al.Load balancing for ultradensenetworks:A deep reinforcement learning-based approach[J].IEEE Internet of Things Journal,2019,6(6):9399-9412.
[15]CHEN X,HU J,CHEN Z,et al.A reinforcement learning-empowered feedback control system for industrial internet of things[J].IEEE Transactions on Industrial Informatics,2021,18(4):2724-2733.
[16]LI J,GAO H,LV T,et al.Deep reinforcement learning basedcomputation offloading and resource allocation for MEC[C]//2018 IEEE Wireless Communications and Networking Confe-rence(WCNC).IEEE,2018:1-6.
[17]ZHANG S,GU H,CHI K,et al.Drl-based partial offloading for maximizing sum computation rate of wireless powered mobile edge computing network[J].IEEE Transactions on Wireless Communications,2022,21(12):10934-10948.
[18]LI K,NI W,YUAN X,et al.Deep-Graph-Based Reinforcement Learning for Joint Cruise Control and Task Offloading for Aerial Edge Internet of Things(EdgeIoT)[J].IEEE Internet of Things Journal,2022,9(21):21676-21686.
[19]CHEN J,YANG Y,WANG C,et al.Multitask offloading strategy optimization based on directed acyclic graphs for edge computing[J].IEEE Internet of Things Journal,2021,9(12):9367-9378.
[20]DAI A,LI R,ZHAO Z,et al.Graph convolutional multi-agentreinforcement learning for UAV coverage control[C]//2020 International Conference on Wireless Communications and Signal Processing(WCSP).IEEE,2020:1106-1111.
[21]ZHANG J,DU J,SHEN Y,et al.Dynamic computation offloa-ding with energy harvesting devices:A hybrid-decision-based deep reinforcement learning approach[J].IEEE Internet of Things Journal,2020,7(10):9303-9317.
[22]GAO Z,YANG L,DAI Y.Fast Adaptive Task Offloading and Resource Allocation in Large-Scale MEC Systems via Multi-Agent Graph Reinforcement Learning[J].IEEE Internet of Things Journal,2023,11(1):758-776.
[23]CAO X,TANG G,GUO D,et al.Edge federation:Towards an integrated service provisioning model[J].IEEE/ACM Transactions on Networking,2020,28(3):1116-1129.
[24]MNIH V,KAVUKCUOGLU K,SILVER D,et al.Human-level control through deep reinforcement learning[J].Nature,2015,518(7540):529-533.
[25]WANG S,ZHAO Y,XU J,et al.Edge server placement in mobile edge computing[J].Journal of Parallel and Distributed Computing,2019,127:160-168.
[26]WANG S,ZHAO Y,HUANG L,et al.QoS prediction for ser-vice recommendations in mobile edge computing[J].Journal of Parallel and Distributed Computing,2019,127:134-144.
[27]GUO Y,WANG S,ZHOU A,et al.User allocation-aware edge cloud placement in mobile edge computing[J].Software:Practice and Experience,2020,50(5):489-502.
[28]WANG S,GUO Y,ZHANG N,et al.Delay-aware microservice coordination in mobile edge computing:A reinforcement learning approach[J].IEEE Transactions on Mobile Computing,2019,20(3):939-951.
[29]VILAPLANA J,SOLSONA F,TEIXIDÓ I,et al.A queuingtheory model for cloud computing[J].The Journal of Supercomputing,2014,69:492-507.
[30]CHEN X,YAO Z,CHEN Z,et al.Load Balancing for Multi-Edge Collaboration in Wireless Metropolitan Area Networks:A Two-Stage Decision-Making Approach[J].IEEE Internet of Things Journal,2023,10(19):17124-17136.
[31]WU Y,GUO K,HUANG J,et al.Secrecy-based energy-efficient data offloading via dual connectivity over unlicensed spectrums[J].IEEE Journal on Selected Areas in Communications,2016,34(12):3252-3270.
[1] LIU Hongjian, ZOU Danping, LI Ping. Pedestrian Trajectory Prediction Method Based on Graph Attention Interaction [J]. Computer Science, 2026, 53(1): 97-103.
[2] WANG Haoyan, LI Chongshou, LI Tianrui. Reinforcement Learning Method for Solving Flexible Job Shop Scheduling Problem Based onDouble Layer Attention Network [J]. Computer Science, 2026, 53(1): 231-240.
[3] CHEN Jintao, LIN Bing, LIN Song, CHEN Jing, CHEN Xing. Dynamic Pricing and Energy Scheduling Strategy for Photovoltaic Storage Charging Stations Based on Multi-agent Deep Reinforcement Learning [J]. Computer Science, 2025, 52(9): 337-345.
[4] LI Yaru, WANG Qianqian, CHE Chao, ZHU Deheng. Graph-based Compound-Protein Interaction Prediction with Drug Substructures and Protein 3D Information [J]. Computer Science, 2025, 52(9): 71-79.
[5] WU Hanyu, LIU Tianci, JIAO Tuocheng, CHE Chao. DHMP:Dynamic Hypergraph-enhanced Medication-aware Model for Temporal Health EventPrediction [J]. Computer Science, 2025, 52(9): 88-95.
[6] XU Jinlong, WANG Gengwu, HAN Lin, NIE Kai, LI Haoran, CHEN Mengyao, LIU Haohao. Research on Parallel Scheduling Strategy Optimization Technology Based on Sunway Compiler [J]. Computer Science, 2025, 52(9): 137-143.
[7] FAN Xinggang, JIANG Xinyang, GU Wenting, XU Juntao, YANG Youdong, LI Qiang. Effective Task Offloading Strategy Based on Heterogeneous Nodes [J]. Computer Science, 2025, 52(8): 354-362.
[8] SU Shiyu, YU Jiong, LI Shu, JIU Shicheng. Cross-domain Graph Anomaly Detection Via Dual Classification and Reconstruction [J]. Computer Science, 2025, 52(8): 374-384.
[9] TANG Boyuan, LI Qi. Review on Application of Spatial-Temporal Graph Neural Network in PM2.5 ConcentrationForecasting [J]. Computer Science, 2025, 52(8): 71-85.
[10] GUO Husheng, ZHANG Xufei, SUN Yujie, WANG Wenjian. Continuously Evolution Streaming Graph Neural Network [J]. Computer Science, 2025, 52(8): 118-126.
[11] ZHANG Yongliang, LI Ziwen, XU Jiahao, JIANG Yuchen, CUI Ying. Congestion-aware and Cached Communication for Multi-agent Pathfinding [J]. Computer Science, 2025, 52(8): 317-325.
[12] LUO Xuyang, TAN Zhiyi. Knowledge-aware Graph Refinement Network for Recommendation [J]. Computer Science, 2025, 52(7): 103-109.
[13] HAO Jiahui, WAN Yuan, ZHANG Yuhang. Research on Node Learning of Graph Neural Networks Fusing Positional and StructuralInformation [J]. Computer Science, 2025, 52(7): 110-118.
[14] JIANG Kun, ZHAO Zhengpeng, PU Yuanyuan, HUANG Jian, GU Jinjing, XU Dan. Cross-modal Hypergraph Optimisation Learning for Multimodal Sentiment Analysis [J]. Computer Science, 2025, 52(7): 210-217.
[15] HUO Dan, YU Fuping, SHEN Di, HAN Xueyan. Research on Multi-machine Conflict Resolution Based on Deep Reinforcement Learning [J]. Computer Science, 2025, 52(7): 271-278.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.