计算机科学

计算机科学 ›› 2025, Vol. 52 ›› Issue (3): 338-348.doi: 10.11896/jsjkx.240100091

• 计算机网络 • 上一篇    下一篇

基于图强化学习的多边缘协同负载均衡方法

郑龙海1,2,3, 肖博怀1,2,3, 姚泽玮1,2,3, 陈星1,2,3, 莫毓昌4   

  1. 1 福州大学计算机与大数据学院 福州 350116
    2 大数据智能教育部工程研究中心 福州 350002
    3 福建省网络计算与智能信息处理重点实验室 福州 350116
    4 华侨大学计算科学福建省高校重点实验室 福建 泉州 362021
  • 收稿日期:2024-01-08 修回日期:2024-05-31 出版日期:2025-03-15 发布日期:2025-03-07
  • 通讯作者: 陈星(chenxing@fzu.edu.cn)
  • 作者简介:(longhai20221108@163.com)
  • 基金资助:
    国家自然科学基金(62072108);福建省科技经济融合服务平台(2023XRH001);福厦泉国家自主创新示范区协同创新平台项目(2022FX5)

Graph Reinforcement Learning Based Multi-edge Cooperative Load Balancing Method

ZHENG Longhai1,2,3, XIAO Bohuai1,2,3, YAO Zewei1,2,3, CHEN Xing1,2,3, MO Yuchang4   

  1. 1 College of Computer and Data Science,Fuzhou University,Fuzhou 350116,China
    2 Engineering Research Center of Big Data Intelligence,Ministry of Education,Fuzhou 350002,China
    3 Fujian Key Laboratory of Network Computing and Intelligent Information Processing,Fuzhou University,Fuzhou 350116,China
    4 Fujian Province University Key Laboratory of Computational Science,Huaqiao University,Quanzhou,Fujian 362021,China
  • Received:2024-01-08 Revised:2024-05-31 Online:2025-03-15 Published:2025-03-07
  • About author:ZHENG Longhai,born in 1999,postgraduate.His main research interests include load balancing and task offloa-ding.
    CHEN Xing,born in 1985,Ph.D,professor,Ph.D supervisor,is a senior member of CCF(No.35725M).His main research interests include software engineering,system software and cloud computing.
  • Supported by:
    National Natural Science Foundation of China(62072108),Fujian Province Technology and Economy Integration Service Platform(2023XRH001) and Fuzhou-Xiamen-Quanzhou National Independent Innovation Demonstration Zone Collaborative Innovation Platform(2022FX5).

PDF (PC)

160

摘要： 在移动边缘计算中,设备通过将计算密集型任务卸载到附近边缘服务器,可以有效减少应用程序的延迟和能耗。为了提高服务质量,边缘服务器之间需要协作而非单独工作。针对多边缘协作的负载均衡问题,现有的策略往往依赖于精确的数学模型或缺乏对边缘拓扑关系的利用。为了解决此问题,文中提出了一种基于图强化学习的卸载决策方法。首先将多边缘协作的负载均衡场景抽象为图数据;然后采用基于图卷积神经网络的图嵌入过程来提取图的信息特征,以辅助深度Q网络进行卸载决策;最后通过集中反馈控制机制找到目标负载均衡方案。在多个场景下进行仿真实验,实验结果验证了所提方法在缩短任务平均响应时延方面的有效性,并且可以在短时间内获得优于对比算法且接近理想方案的负载均衡效果。

关键词: 多边缘协作, 负载均衡, 任务卸载, 图神经网络, 深度强化学习

Abstract: In mobile edge computing,devices can effectively relieve latency and energy consumption by offloading computation-intensive tasks to nearby edge servers.In order to improve the quality of service,edge servers need to collaborate with each other rather than working alone.For the load balancing problem of multi-edge collaboration,the existing solutions often depend on accurate mathematical models or make fair use of edge topological relationships.To solve this problem,an offloading decision-ma-king method based on graph reinforcement learning is proposed in this paper.Firstly,the load balancing scenario with multi-edge collaboration is abstracted as graph data,then a graph embedding process based on graph convolutional neural network is used to extract the information features of the graph,for assisting the deep Q-network to make offloading decisions,and finally the objective load balancing plan is found through a centralized feedback-control mechanism.Simulation experiments are conducted in multiple scenarios,the results verify the effectiveness of the proposed method in shortening the average response latency of the tasks,and the load balancing effect which is better than the comparison algorithms and close to the ideal plan can be obtained in a short period of time.

Key words: Multi-edge collaboration, Load balancing, Task offloading, Graph neural network, Deep reinforcement learning

中图分类号: 

  • TP338
[1]YAO Z,LIN J,HU J,et al.PSO-GA based approach to multi-edge load balancing[J].Computer Science,2021,48(S2):456-463.
[2]XU J,CHEN L,REN S.Online learning for offloading and autoscaling in energy harvesting mobile edge computing[J].IEEE Transactions on Cognitive Communications and Networking,2017,3(3):361-373.
[3]ALASMARI K R,GREEN R C,ALAM M.Mobile edge offloa-ding using markov decision processes[C]//Edge Computing-EDGE 2018:Second International Conference,Held as Part of the Services Conference Federation,SCF 2018,Seattle,WA,USA,June 25－30,2018,Proceedings 2.Springer International Publishing,2018:80-90.
[4]SUN Z,MO Y,YU C.Graph reinforcement learning based taskoffloading for multi-access edge computing[J].IEEE Internet of Things Journal,2023,10(4):3138-3150.
[5]CABRERA A,ACOSTA A,ALMEIDA F,et al.A dynamicmulti-objective approach for dynamic load balancing in heterogeneous systems[J].IEEE Transactions on Parallel and Distributed Systems,2020,31(10):2421-2434.
[6]MEHRABI A,SIEKKINEN M,YLÄ-JÄÄSKI A.Edge computing assisted adaptive mobile video streaming[J].IEEE Transactions on Mobile Computing,2018,18(4):787-800.
[7]WANG Z,ZHANG W,JIN X,et al.An optimal edge serverplacement approach for cost reduction and load balancing in intelligent manufacturing[J].The Journal of Supercomputing,2022,78(3):4032-4056.
[8]YANG L,YAO H,WANG J,et al.Multi-UAV-enabled load-balance mobile-edge computing for IoT networks[J].IEEE Internet of Things Journal,2020,7(8):6898-6908.
[9]JIA M,LIANG W,XU Z,et al.Qos-aware cloudlet load balancing in wireless metropolitan area networks[J].IEEE Transactions on Cloud Computing,2018,8(2):623-634.
[10]TRAN T X,POMPILI D.Joint task offloading and resource allocation for multi-server mobile-edge computing networks[J].IEEE Transactions on Vehicular Technology,2018,68(1):856-868.
[11]XU X,LIU K,DAI P,et al.Joint task offloading and resource optimization in noma-based vehicular edge computing:A game-theoretic drl approach[J].Journal of Systems Architecture,2023,134:102780.
[12]LIU Q,XIA T,CHENG L,et al.Deep reinforcement learning for load-balancing aware network control in IoT edge systems[J].IEEE Transactions on Parallel and Distributed Systems,2021,33(6):1491-1502.
[13]LI J,LUO G,CHENG N,et al.An end-to-end load balancer based on deep learning for vehicular network traffic control[J].IEEE Internet of Things Journal,2018,6(1):953-966.
[14]XU Y,XU W,WANG Z,et al.Load balancing for ultradensenetworks:A deep reinforcement learning-based approach[J].IEEE Internet of Things Journal,2019,6(6):9399-9412.
[15]CHEN X,HU J,CHEN Z,et al.A reinforcement learning-empowered feedback control system for industrial internet of things[J].IEEE Transactions on Industrial Informatics,2021,18(4):2724-2733.
[16]LI J,GAO H,LV T,et al.Deep reinforcement learning basedcomputation offloading and resource allocation for MEC[C]//2018 IEEE Wireless Communications and Networking Confe-rence(WCNC).IEEE,2018:1-6.
[17]ZHANG S,GU H,CHI K,et al.Drl-based partial offloading for maximizing sum computation rate of wireless powered mobile edge computing network[J].IEEE Transactions on Wireless Communications,2022,21(12):10934-10948.
[18]LI K,NI W,YUAN X,et al.Deep-Graph-Based Reinforcement Learning for Joint Cruise Control and Task Offloading for Aerial Edge Internet of Things(EdgeIoT)[J].IEEE Internet of Things Journal,2022,9(21):21676-21686.
[19]CHEN J,YANG Y,WANG C,et al.Multitask offloading strategy optimization based on directed acyclic graphs for edge computing[J].IEEE Internet of Things Journal,2021,9(12):9367-9378.
[20]DAI A,LI R,ZHAO Z,et al.Graph convolutional multi-agentreinforcement learning for UAV coverage control[C]//2020 International Conference on Wireless Communications and Signal Processing(WCSP).IEEE,2020:1106-1111.
[21]ZHANG J,DU J,SHEN Y,et al.Dynamic computation offloa-ding with energy harvesting devices:A hybrid-decision-based deep reinforcement learning approach[J].IEEE Internet of Things Journal,2020,7(10):9303-9317.
[22]GAO Z,YANG L,DAI Y.Fast Adaptive Task Offloading and Resource Allocation in Large-Scale MEC Systems via Multi-Agent Graph Reinforcement Learning[J].IEEE Internet of Things Journal,2023,11(1):758-776.
[23]CAO X,TANG G,GUO D,et al.Edge federation:Towards an integrated service provisioning model[J].IEEE/ACM Transactions on Networking,2020,28(3):1116-1129.
[24]MNIH V,KAVUKCUOGLU K,SILVER D,et al.Human-level control through deep reinforcement learning[J].Nature,2015,518(7540):529-533.
[25]WANG S,ZHAO Y,XU J,et al.Edge server placement in mobile edge computing[J].Journal of Parallel and Distributed Computing,2019,127:160-168.
[26]WANG S,ZHAO Y,HUANG L,et al.QoS prediction for ser-vice recommendations in mobile edge computing[J].Journal of Parallel and Distributed Computing,2019,127:134-144.
[27]GUO Y,WANG S,ZHOU A,et al.User allocation-aware edge cloud placement in mobile edge computing[J].Software:Practice and Experience,2020,50(5):489-502.
[28]WANG S,GUO Y,ZHANG N,et al.Delay-aware microservice coordination in mobile edge computing:A reinforcement learning approach[J].IEEE Transactions on Mobile Computing,2019,20(3):939-951.
[29]VILAPLANA J,SOLSONA F,TEIXIDÓ I,et al.A queuingtheory model for cloud computing[J].The Journal of Supercomputing,2014,69:492-507.
[30]CHEN X,YAO Z,CHEN Z,et al.Load Balancing for Multi-Edge Collaboration in Wireless Metropolitan Area Networks:A Two-Stage Decision-Making Approach[J].IEEE Internet of Things Journal,2023,10(19):17124-17136.
[31]WU Y,GUO K,HUANG J,et al.Secrecy-based energy-efficient data offloading via dual connectivity over unlicensed spectrums[J].IEEE Journal on Selected Areas in Communications,2016,34(12):3252-3270.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市渝北区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发