计算机科学

计算机科学 ›› 2024, Vol. 51 ›› Issue (11A): 231100170-10.doi: 10.11896/jsjkx.231100170

• 网络&通信 • 上一篇    下一篇

基于深度强化学习的云边协同任务迁移与资源再分配优化研究

陈娟1, 王阳1, 吴宗玲2, 陈鹏1, 张逢春1, 郝俊峰1   

  1. 1 西华大学计算机与软件工程学院 成都 610039
    2 西南交通大学信息科学与技术学院 成都 611756
  • 出版日期:2024-11-16 发布日期:2024-11-13
  • 通讯作者: 陈鹏(chenpeng@mail.xhu.edu.cn)
  • 作者简介:(chenjuan@mail.xhu.edu.cn)
  • 基金资助:
    国家自然科学基金(62376043);四川省科技计划(2020JDRC0067,2023JDRC0087);四川省自然科学基金(2024NSFTD008,2022NSFSC0556);教育部春晖计划(HZKY20220578)

Cloud-Edge Collaborative Task Transfer and Resource Reallocation Optimization Based on Deep Reinforcement Learning

CHEN Juan1, WANG Yang1, WU Zongling2, CHEN Peng1, ZHANG Fengchun1 , HAO Junfeng1   

  1. 1 School of Computer and Software Engineering,Xihua University,Chengdu 610039,China
    2 School of Information Science and Technology,Southwest Jiaotong University,Chengdu 611756,China
  • Online:2024-11-16 Published:2024-11-13
  • About author:CHEN Juan,born in 1985,Ph.D,asso-ciate professor.Her main research in-terests include cloud computing,mobile-edge computing,and deep reinforcement learning.
    CHEN Peng,born in 1979,Ph.D.His main research interests include cloud computing,machine learning,and deep learning.
  • Supported by:
    National Natural Science Foundation of China (62376043),Science and Technology Program of Sichuan Province(2020JDRC0067,2023JDRC0087),Natural Science Foundation of Sichuan Province of China(2024NSFTD008,2022NSFSC0556)and Chunhui Program of Ministry of Education of China(HZKY20220578).

PDF (PC)

178

摘要： 文中研究了一个由多个边缘服务器和云服务器组成的异构云边环境,每个节点都具有计算、存储和通信能力。由于异构云边环境的不确定性和动态性,需要进行动态调度以优化资源和任务分配。传统的深度学习框架只从输入的任务数据中提取潜在的特征,大多数忽略了云边环境的网络结构信息。为了解决这个问题,文中基于Actor-Critic框架,利用柔性动作-评价(Soft Actor-Critic,SAC)经验训练的自进化能力和图卷积网络(Graph Convolutional Networks,GCN)中基于图的关系推演能力,提出了一种集中式的SAC-GCN算法。提出的SAC-GCN通过捕获动态的任务信息和异构的节点信息,采用自适应的损失函数来提供有效的调度策略,以适应不同的任务迁移需求。文中采用来源于真实世界的Bit-brain数据集,并通过Cloud-Sim进行大量模拟。实验结果表明,与现有算法相比,提出的SAC-GCN可以减少4.81%系统能耗,缩短3.46%任务响应时间和2.73%的任务迁移时间,以及减小1.5%的任务SLA违规率。

关键词: 云计算, 边缘计算, 深度强化学习, 动态调度, 柔性动作-评价, 图卷积

Abstract: In this paper,we have investigated a heterogeneous cloud-edge environment consisting of multiple edge servers and cloud servers,where each node has computation,storage and communication capabilities.Due to the uncertainty and dynamics of the heterogeneous cloud edge environment,dynamic scheduling is required to optimize resource and task allocation.The traditional deep learning framework only extract the potential features from the input task data,mostly ignoring the network structure information characteristics of the cloud-edge environment.To solve this problem,this paper proposes a distributed SAC-GCN algorithm based on the Actor-Critic framework,using the self-evolutionary ability of the experience training of soft actor-critic(SAC) and the graph-based relationship inference ability of graph convolutional networks(GCN).The proposed SAC-GCN employs an adaptive loss function to provide effective scheduling strategies for different task migration requirements by capturing dynamic task information and heterogeneous node resource information.In this paper,we utilize the Bit-brain dataset sourced from the real world,and carries out a large number of simulations through Cloud-Sim.Experimental results show that compared with the exis-ting algorithms,the proposed SAC-GCN can reduce the system energy consumption by 4.81%,shorten the task response time by 3.46% and the task migration time by 2.73%,and reduce the task SLA violation rate by 1.5%.

Key words: Cloud computing, Edge computing, Deep reinforcement learning, Dynamic scheduling, Flexible action-evaluation, Graph convolution

中图分类号: 

  • TP18
[1]MAHMUD R,SRIRAMA S,RAMAMOHANARAO K,et al.Quality of experience(QoE)-aware placement of applications in fog computing environments[J].Journal of Parallel and Distri-buted Computing,2019,132:190-203.
[2]GAO H,XU Y,YIN Y,et al.Context-aware QoS predictionwith neural collaborative filtering for Internet-of-Things services[J].IEEE Internet of Things Journal,2020,7(5):4532-4542.
[3]JIA M,LIANG W,XU Z,et al.Qos-aware cloudlet load balancing in wireless metropolitan area networks[J].IEEE Transactions on Cloud Computing,2020,8(2):623-634.
[4]SHEN H,CHEN L.A resource-efficient predictive resourceProvision-ing System in Cloud Systems[J].IEEE Transactions on Parallel and Distributed Systems,2022,33(12):3886-3900.
[5]HADDADI M,BAHNES N.Well-known open-source cloudcomputing platforms[C]//Proceed-ings of the International Conference on Infor-mation Systems and Advanced Technologies.Tebessa,Algeria,2021:1-6.
[6]GU L,ZHANG W,WANG Z,et al.Service Management andEnergy Scheduling Toward Low-Carbon Edge Computing[J].IEEE Transactions on Sustainable Computing,2023,8(1):109-119.
[7]HARJULA E,ARTEMENKO A,FORSSTRÖM S.Edge com-puting for in-dustrial IoT:Challenges and Solutions[M]//Wireless Networks and Industrial IoT.Cham:Springer,2021:225-240.
[8]CHEN Y,LIU B,LIN W,et al.Survey of Cloud-edge Collaboration[J].Computer Science,2021,48(3):259-268.
[9]CAI X,GENG S,WU D,et al.A multicloud-model-based many-objective intelligent algorithm for efficient task scheduling in internet of things[J].IEEE Internet of Things Journal,2021,8(12):9645-9653.
[10]ALSULAMI A,AL-HAIJA Q,THANOONM,et al.Perfor-mance evaluation of dynamic round robin algorithms for CPU scheduling[C]//Proceedings of the SoutheastCon.Huntsville,AL,USA,2019:1-5.
[11]LIU S,WANG N.Collaborative Optimization Scheduling ofCloud Service Resources Based on Improved Genetic Algorithm[J].IEEE Access,2020,8:150878-150890.
[12]YUAN H,WANG Y.A Hybrid Schedule Technology Based on Genetic Algorithm and Simulated Annealing for Time-Triggered Ethernet[C]//Proceedings of the IEEE 2nd International Conference on Information Communication and Software Enginee-ring.Chongqing,China,2022:151-155.
[13]JIA J,WANG W.Review of reinforcement learning research[C]//Proceedings of the 35th Youth Academic Annual Confe-rence of Chinese Association of Automation.Zhanjiang,China,2020:186-191.
[14]TULI S,ILAGER S,RAMAMOHANARAOK,et al.Dynamicscheduling for stochastic edge-cloud computing environments using a3c learning and residual recurrent neural networks[J].IEEE Transactions on Mobile Computing,2022,21(3):940-954.
[15]TUOMAS H,AURICK Z,PIETER A,et al.Soft actor-critic:off-policy maximum entropy deep reinforcement learning with a stochastic actor[C]//Proceedings of the 35th International Conference on Machine Learning.Stockholmsmässan,Stockholm Sweden,2018:1861-1870.
[16]XU B,CEN K,HUANG J,et al.Survey on graph convolutional network[J].Chinese Journal of Computers,2020,43(5):755-780.
[17]BELOGLAZOV A,BUYYA R.Optimal online deterministic algorithms and adaptive heuristics for energy and performance efficient dynamic consolidation of virtual machines in cloud datacenters[J].Concurrency and Computation:Practice and Expe-rience,2011,24(13):1397-1420.
[18]ALIYAH Z,PAMBUDHI R,AHNAFI H,et al.Comparison of earliest deadline first and rate monotonic scheduling in polling server[C]//Proceedings of the 8th International Conference on Information and Communication Technology.Yogyakarta,Indonesia,2020:1-4.
[19]GAO M,ZHU Y,SUN J.The multi-objective cloud tasks sche-duling based on hybrid particle swarm optimization[C]//Proceedings of the Eighth International Conference on Advanced Cloud and Big Data.Taiyuan,China,2020:1-5.
[20]XUE N,DING D,FAN Y,et al.Research on Joint Scheduling Method of Heterogeneous TT&C Network Resources Based on Improved DQN Algorithm[C]//Proceedings of the 2nd International Conference on Information Technology,Big Data and Artificial Intelligence.Chongqing,China,2021:1009-1014.
[21]HU S,SONG R,CHEN X,et al.Dependency-Task Scheduling in Cloud-Edge Collaborative Computing Based on Reinforcement Learning[J].Computer Science,2023,50(11A):220900076-8.
[22]CHEN J,XING H,XIAO Z,et al.A DRL agent for jointly optimizing computation offloading and resource allocation in MEC[J].IEEE Internet of Things Journal,2021,8(24):17508-17524.
[23]CHEN J,WU Z.Dynamic computation offloading with energyharvesting devices:a graph-based deep reinforcement learning approach[J].IEEE Communications Letters,2021,25(9):2968-2972.
[24]HAARNOJA T,TANG H,ABBEEL P,et al.Reinforcementlearning with deep energy-based policies[C]//Proceedings of the 34th International Conference on Machine Learning.Sydney NSW Australia,2017:1352-1361.
[25]SUNDAS A,PANDA S.An introduction of cloudsim simulation tool for modelling and scheduling[C]//Proceedings of the International Conference on Emerging Smart Computing and Informatics.Pune,India,2020:263-268.
[26]SUPREETH S,PATIL K,PATIL S,et al.Comparative ap-proach for VM scheduling using modified particle swarm optimization and genetic algorithm in cloud computing[C]//Procee-dings of the IEEE International Conference on Data Science and Information System.Hassan,India,2022:1-6
[27]CHENG M,LI J,NAZARIAN S D R.L-cloud:deep reinforcement learning-based resource pro-visioning and task scheduling for cloud service providers[C]//Proceedings of the 23rd Asia and South Pacific Design Automation Conference.Jeju,Korea(South),2018:129-134.
[28]BASU D,WANG X,HONG Y,et al.Learn-as-you-go withmegh:efficient live migration of virtual machines[J].IEEE Transactions on Parallel and Distributed Systems,2019,30(8):1786-1801.
[29]SHEN S,VAN V,IOSUP A.Statistical characterization of business-critical workloads hosted in cloud datacenters[C]//Proceedings of the 15th IEEE/ACM International Symposium on Cluster,Cloud and Grid Computing.Shenzhen,China,2015:465-474.
[30]MAOH,ALIZADEH M,MENACHE I,et al.Resource management with deep reinforce-ment learning[C]//Proceedings of the 15th ACM Work-shop on Hot Topics in Networks.Atlanta GA USA,2016:50-56.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市渝北区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发