Computer Science

Computer Science ›› 2024, Vol. 51 ›› Issue (11A): 231100170-10.doi: 10.11896/jsjkx.231100170

• Network & Communication • Previous Articles     Next Articles

Cloud-Edge Collaborative Task Transfer and Resource Reallocation Optimization Based on Deep Reinforcement Learning

CHEN Juan1, WANG Yang1, WU Zongling2, CHEN Peng1, ZHANG Fengchun1 , HAO Junfeng1   

  1. 1 School of Computer and Software Engineering,Xihua University,Chengdu 610039,China
    2 School of Information Science and Technology,Southwest Jiaotong University,Chengdu 611756,China
  • Online:2024-11-16 Published:2024-11-13
  • About author:CHEN Juan,born in 1985,Ph.D,asso-ciate professor.Her main research in-terests include cloud computing,mobile-edge computing,and deep reinforcement learning.
    CHEN Peng,born in 1979,Ph.D.His main research interests include cloud computing,machine learning,and deep learning.
  • Supported by:
    National Natural Science Foundation of China (62376043),Science and Technology Program of Sichuan Province(2020JDRC0067,2023JDRC0087),Natural Science Foundation of Sichuan Province of China(2024NSFTD008,2022NSFSC0556)and Chunhui Program of Ministry of Education of China(HZKY20220578).

PDF (PC)

178

Abstract: In this paper,we have investigated a heterogeneous cloud-edge environment consisting of multiple edge servers and cloud servers,where each node has computation,storage and communication capabilities.Due to the uncertainty and dynamics of the heterogeneous cloud edge environment,dynamic scheduling is required to optimize resource and task allocation.The traditional deep learning framework only extract the potential features from the input task data,mostly ignoring the network structure information characteristics of the cloud-edge environment.To solve this problem,this paper proposes a distributed SAC-GCN algorithm based on the Actor-Critic framework,using the self-evolutionary ability of the experience training of soft actor-critic(SAC) and the graph-based relationship inference ability of graph convolutional networks(GCN).The proposed SAC-GCN employs an adaptive loss function to provide effective scheduling strategies for different task migration requirements by capturing dynamic task information and heterogeneous node resource information.In this paper,we utilize the Bit-brain dataset sourced from the real world,and carries out a large number of simulations through Cloud-Sim.Experimental results show that compared with the exis-ting algorithms,the proposed SAC-GCN can reduce the system energy consumption by 4.81%,shorten the task response time by 3.46% and the task migration time by 2.73%,and reduce the task SLA violation rate by 1.5%.

Key words: Cloud computing, Edge computing, Deep reinforcement learning, Dynamic scheduling, Flexible action-evaluation, Graph convolution

CLC Number: 

  • TP18
[1]MAHMUD R,SRIRAMA S,RAMAMOHANARAO K,et al.Quality of experience(QoE)-aware placement of applications in fog computing environments[J].Journal of Parallel and Distri-buted Computing,2019,132:190-203.
[2]GAO H,XU Y,YIN Y,et al.Context-aware QoS predictionwith neural collaborative filtering for Internet-of-Things services[J].IEEE Internet of Things Journal,2020,7(5):4532-4542.
[3]JIA M,LIANG W,XU Z,et al.Qos-aware cloudlet load balancing in wireless metropolitan area networks[J].IEEE Transactions on Cloud Computing,2020,8(2):623-634.
[4]SHEN H,CHEN L.A resource-efficient predictive resourceProvision-ing System in Cloud Systems[J].IEEE Transactions on Parallel and Distributed Systems,2022,33(12):3886-3900.
[5]HADDADI M,BAHNES N.Well-known open-source cloudcomputing platforms[C]//Proceed-ings of the International Conference on Infor-mation Systems and Advanced Technologies.Tebessa,Algeria,2021:1-6.
[6]GU L,ZHANG W,WANG Z,et al.Service Management andEnergy Scheduling Toward Low-Carbon Edge Computing[J].IEEE Transactions on Sustainable Computing,2023,8(1):109-119.
[7]HARJULA E,ARTEMENKO A,FORSSTRÖM S.Edge com-puting for in-dustrial IoT:Challenges and Solutions[M]//Wireless Networks and Industrial IoT.Cham:Springer,2021:225-240.
[8]CHEN Y,LIU B,LIN W,et al.Survey of Cloud-edge Collaboration[J].Computer Science,2021,48(3):259-268.
[9]CAI X,GENG S,WU D,et al.A multicloud-model-based many-objective intelligent algorithm for efficient task scheduling in internet of things[J].IEEE Internet of Things Journal,2021,8(12):9645-9653.
[10]ALSULAMI A,AL-HAIJA Q,THANOONM,et al.Perfor-mance evaluation of dynamic round robin algorithms for CPU scheduling[C]//Proceedings of the SoutheastCon.Huntsville,AL,USA,2019:1-5.
[11]LIU S,WANG N.Collaborative Optimization Scheduling ofCloud Service Resources Based on Improved Genetic Algorithm[J].IEEE Access,2020,8:150878-150890.
[12]YUAN H,WANG Y.A Hybrid Schedule Technology Based on Genetic Algorithm and Simulated Annealing for Time-Triggered Ethernet[C]//Proceedings of the IEEE 2nd International Conference on Information Communication and Software Enginee-ring.Chongqing,China,2022:151-155.
[13]JIA J,WANG W.Review of reinforcement learning research[C]//Proceedings of the 35th Youth Academic Annual Confe-rence of Chinese Association of Automation.Zhanjiang,China,2020:186-191.
[14]TULI S,ILAGER S,RAMAMOHANARAOK,et al.Dynamicscheduling for stochastic edge-cloud computing environments using a3c learning and residual recurrent neural networks[J].IEEE Transactions on Mobile Computing,2022,21(3):940-954.
[15]TUOMAS H,AURICK Z,PIETER A,et al.Soft actor-critic:off-policy maximum entropy deep reinforcement learning with a stochastic actor[C]//Proceedings of the 35th International Conference on Machine Learning.Stockholmsmässan,Stockholm Sweden,2018:1861-1870.
[16]XU B,CEN K,HUANG J,et al.Survey on graph convolutional network[J].Chinese Journal of Computers,2020,43(5):755-780.
[17]BELOGLAZOV A,BUYYA R.Optimal online deterministic algorithms and adaptive heuristics for energy and performance efficient dynamic consolidation of virtual machines in cloud datacenters[J].Concurrency and Computation:Practice and Expe-rience,2011,24(13):1397-1420.
[18]ALIYAH Z,PAMBUDHI R,AHNAFI H,et al.Comparison of earliest deadline first and rate monotonic scheduling in polling server[C]//Proceedings of the 8th International Conference on Information and Communication Technology.Yogyakarta,Indonesia,2020:1-4.
[19]GAO M,ZHU Y,SUN J.The multi-objective cloud tasks sche-duling based on hybrid particle swarm optimization[C]//Proceedings of the Eighth International Conference on Advanced Cloud and Big Data.Taiyuan,China,2020:1-5.
[20]XUE N,DING D,FAN Y,et al.Research on Joint Scheduling Method of Heterogeneous TT&C Network Resources Based on Improved DQN Algorithm[C]//Proceedings of the 2nd International Conference on Information Technology,Big Data and Artificial Intelligence.Chongqing,China,2021:1009-1014.
[21]HU S,SONG R,CHEN X,et al.Dependency-Task Scheduling in Cloud-Edge Collaborative Computing Based on Reinforcement Learning[J].Computer Science,2023,50(11A):220900076-8.
[22]CHEN J,XING H,XIAO Z,et al.A DRL agent for jointly optimizing computation offloading and resource allocation in MEC[J].IEEE Internet of Things Journal,2021,8(24):17508-17524.
[23]CHEN J,WU Z.Dynamic computation offloading with energyharvesting devices:a graph-based deep reinforcement learning approach[J].IEEE Communications Letters,2021,25(9):2968-2972.
[24]HAARNOJA T,TANG H,ABBEEL P,et al.Reinforcementlearning with deep energy-based policies[C]//Proceedings of the 34th International Conference on Machine Learning.Sydney NSW Australia,2017:1352-1361.
[25]SUNDAS A,PANDA S.An introduction of cloudsim simulation tool for modelling and scheduling[C]//Proceedings of the International Conference on Emerging Smart Computing and Informatics.Pune,India,2020:263-268.
[26]SUPREETH S,PATIL K,PATIL S,et al.Comparative ap-proach for VM scheduling using modified particle swarm optimization and genetic algorithm in cloud computing[C]//Procee-dings of the IEEE International Conference on Data Science and Information System.Hassan,India,2022:1-6
[27]CHENG M,LI J,NAZARIAN S D R.L-cloud:deep reinforcement learning-based resource pro-visioning and task scheduling for cloud service providers[C]//Proceedings of the 23rd Asia and South Pacific Design Automation Conference.Jeju,Korea(South),2018:129-134.
[28]BASU D,WANG X,HONG Y,et al.Learn-as-you-go withmegh:efficient live migration of virtual machines[J].IEEE Transactions on Parallel and Distributed Systems,2019,30(8):1786-1801.
[29]SHEN S,VAN V,IOSUP A.Statistical characterization of business-critical workloads hosted in cloud datacenters[C]//Proceedings of the 15th IEEE/ACM International Symposium on Cluster,Cloud and Grid Computing.Shenzhen,China,2015:465-474.
[30]MAOH,ALIZADEH M,MENACHE I,et al.Resource management with deep reinforce-ment learning[C]//Proceedings of the 15th ACM Work-shop on Hot Topics in Networks.Atlanta GA USA,2016:50-56.
[1] WANG Tianjiu, LIU Quan, WU Lan. Offline Reinforcement Learning Algorithm for Conservative Q-learning Based on Uncertainty Weight [J]. Computer Science, 2024, 51(9): 265-272.
[2] ZHOU Wenhui, PENG Qinghua, XIE Lei. Study on Adaptive Cloud-Edge Collaborative Scheduling Methods for Multi-object State Perception [J]. Computer Science, 2024, 51(9): 319-330.
[3] LI Zhi, LIN Sen, ZHANG Qiang. Edge Cloud Computing Approach for Intelligent Fault Detection in Rail Transit [J]. Computer Science, 2024, 51(9): 331-337.
[4] ZHANG Lu, DUAN Youxiang, LIU Juan, LU Yuxi. Chinese Geological Entity Relation Extraction Based on RoBERTa and Weighted Graph Convolutional Networks [J]. Computer Science, 2024, 51(8): 297-303.
[5] YUAN Lining, FENG Wengang, LIU Zhao. Multi-channel Graph Convolutional Networks Enhanced by Label Propagation Algorithm [J]. Computer Science, 2024, 51(8): 304-312.
[6] GAO Yuzhao, NIE Yiming. Survey of Multi-agent Deep Reinforcement Learning Based on Value Function Factorization [J]. Computer Science, 2024, 51(6A): 230300170-9.
[7] HUANG Haixin, CAI Mingqi, WANG Yuyao. Review of Point Cloud Semantic Segmentation Based on Graph Convolutional Neural Networks [J]. Computer Science, 2024, 51(6A): 230400196-7.
[8] HUANG Rui, XU Ji. Text Classification Based on Invariant Graph Convolutional Neural Networks [J]. Computer Science, 2024, 51(6A): 230900018-5.
[9] WEI Niannian, HAN Shuguang. New Solution for Traveling Salesman Problem Based on Graph Convolution and AttentionNeural Network [J]. Computer Science, 2024, 51(6A): 230700222-8.
[10] WANG Tian, SHEN Wei, ZHANG Gongxuan, XU Linli, WANG Zhen, YUN Yu. Soft Real-time Cloud Service Request Scheduling and Multiserver System Configuration for ProfitOptimization [J]. Computer Science, 2024, 51(6A): 230900099-10.
[11] WANG Shuanqi, ZHAO Jianxin, LIU Chi, WU Wei, LIU Zhao. Fuzz Testing Method of Binary Code Based on Deep Reinforcement Learning [J]. Computer Science, 2024, 51(6A): 230800078-7.
[12] SUN Jianming, ZHAO Mengxin. Survey of Application of Differential Privacy in Edge Computing [J]. Computer Science, 2024, 51(6A): 230700089-9.
[13] LIU Hui, JI Ke, CHEN Zhenxiang, SUN Runyuan, MA Kun, WU Jun. Malicious Attack Detection in Recommendation Systems Combining Graph Convolutional Neural Networks and Ensemble Methods [J]. Computer Science, 2024, 51(6A): 230700003-9.
[14] XUE Jianbin, DOU Jun, WANG Tao, MA Yuling. Scheme for Maximizing Secure Communication Capacity in UAV-assisted Edge Computing Networks [J]. Computer Science, 2024, 51(6A): 230800032-7.
[15] ZHANG Xiaoxi, LI Dongxi. Cancer Subtype Prediction Based on Similar Network Fusion Algorithm [J]. Computer Science, 2024, 51(6A): 230500006-7.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.