Computer Science

Computer Science ›› 2024, Vol. 51 ›› Issue (5): 293-305.doi: 10.11896/jsjkx.230200121

• Computer Network • Previous Articles     Next Articles

COURIER:Edge Computing Task Scheduling and Offloading Method Based on Non-preemptivePriorities Queuing and Prioritized Experience Replay DRL

YANG Xiuwen1,2,3, CUI Yunhe1,2,3, QIAN Qing4, GUO Chun1,2,3, SHEN Guowei1,2,3   

  1. 1 College of Computer Science and Technology,Guizhou University,Guiyang 550025,China
    2 State Key Laboratory of Public Big Data,Guiyang 550025,China
    3 Engineering Research Center of Text Computing & Cognitive Intelligence,Ministry of Education,Guiyang 550025,China
    4 School of Information,Guizhou University of Finance and Economics,Guiyang 550000,China
  • Received:2023-02-17 Revised:2023-07-29 Online:2024-05-15 Published:2024-05-08
  • About author:YANG Xiuwen,born in 1994,postgra-duate,is a member of CCF(No.T4489).His main research interests include computational offloading and deep reinforcement learning.
    CUI Yunhe,born in 1987,Ph.D,asso-ciate professor,is a member of CCF(No.F3600M).His main research interests include edge computing,network security,software-defined networks,data center networks and network telemetry.
  • Supported by:
    National Natural Science Foundation of China(62102111),Guizhou Provincial Science and Technology Plan([2020]1Y267,Qian Ke He Zhongda Zhuanxiang Zi[2024]003),Natural Science Research Project of Education Department of Guizhou Province([2021136]) and Scientific Research Foundation for Introduced Talents of Guizhou University((2019)52).

PDF (PC)

305

Abstract: Edge computing(EC) deploy a large number of computing and storage resources at the edge of the network to meet requirements on latency and power consumption of tasks.Computing offloading is one of the key technologies in EC.When estimating the delay of task queuing,the existing computation offloading methods usually use M/M/1/∞/∞/FCFS or M/M/n/∞/∞/FCFS models.Without considering the priority of high delay sensitive tasks,these methods cause some computation tasks that do not require sensitive delay always occupy the computation resources,increasing the delay cost of these methods.Meanwhile,most of the existing playback methods use random sampling to replay experience,which cannot distinguish the pros and cons of expe-rience,resulting in low experience utilization and slow neural network convergence.At last,the deterministic policy deep reinforcement learning(DRL) based on computational offloading methods have problems,such as weak ability of exploring environment,low robustness and low experience utilization rate,which reduces the accuracy of solving computational unload problem.To solve the above problems,considering the multi-task mobile device and multi-edge server computing offload scenarios,aims to minimize the system delay and energy consumption,study task scheduling and offloading decision-making problems,and computation offloading qUeuing and pRioritIzed experience replay DRL(COURIER) is proposed.COURIER first designs a non-preemptive priority queuing model(M/M/n/∞/∞/NPR) to optimize the queuing delay of tasks.Then,it proposes a maximum entropy deep reinforcement learning algorithm based on prioritized experience replay.For the offloading decision problem,an offloading decision mechanism of priority experience replay SAC is proposed,based on soft actor-critic(SAC) algorithm.In this mechanism,information entropy is added to the objective function to make the agent adopt random strategy,and the empirical sampling me-thod is optimized to accelerate the convergence rate of the network.Simulation results show that COURIER can effectively reduce system delay and energy consumption.

Key words: Edge computing, Computing offloading, Non-preemptive priority queuing, Information entropy, Deep reinforcement learning, Priority experience replay

CLC Number: 

  • TP391
[1]SPINELLIF,MANCUSO V.Toward enabled industrial verticals in 5G:A survey on MEC-based approaches to provisioning and flexibility[J].IEEE Communications Surveys & Tutorials,2020,23(1):596-630.
[2]SATYANARAYANAN M.The emergence of edge computing[J].Computer,2017,50(1):30-39.
[3]MIAO W W,WANG C J,ZENG Z,et al.An elastic resource scheduling algorithm for online applications from multiple IoT agents[J].Journal of Chongqing University of Technology(Natural Science),2022,36(2):151-161.
[4]ZHANG J,ZHOU Z,LI S,et al.Hybrid computation offloading for smart home automation in mobile cloud computing[J].Personal and Ubiquitous Computing,2018,22:121-134.
[5]MOLOKOMMED N,ONUMANYI A J,ABU-MAHFOUZ AM.Edge intelligence in Smart Grids:A survey on architectures,offloading models,cyber security measures,and challenges[J].Journal of Sensor and Actuator Networks,2022,11(3):47.
[6]YOUSEFPOURA,PATIL A,ISHIGAKI G,et al.Fogplan:Alightweight qos-aware dynamic fog service provisioning framework[J].IEEE Internet of Things Journal,2019,6(3):5080-5096.
[7]XU X,HUANG Q,YIN X,et al.Intelligent offloading for colla-borative smart city services in edge computing[J].IEEE Internet of Things Journal,2020,7(9):7919-7927.
[8]XUE J,WANG Z,ZHANG Y,et al.Task allocation optimiza-tion scheme based on queuing theory for mobile edge computing in 5G heterogeneous networks[J].Mobile Information Systems,2020,2020:1-12.
[9]KUANG Z F,CHEN Q L.Multi-user Edge Computing Task off-loading Scheduling and Resource Allocation Based on Deep Reinforcement Learning[J].Chinese Journal of Computers,2022,45(2):812-824.
[10]WANG Y,GE H,FENG A,et al.Computation offloading stra-tegy based on deep reinforcement learning in cloud-assisted mobile edge computing[C]//2020 IEEE 5th International Confe-rence on Cloud Computing and Big Data Analytics(ICCCBDA).IEEE,2020:108-113.
[11]DAI Y,ZHANG K,MAHARJIA S,et al.Edge intelligence for energy-efficient computation offloading and resource allocation in 5G beyond[J].IEEE Transactions on Vehicular Technology,2020,69(10):12175-12186.
[12]HUANG L,FENG X,ZHANG C,et al.Deep reinforcementlearning-based joint task offloading and bandwidth allocation for multi-user mobile edge computing[J].Digital Communications and Networks,2019,5(1):10-17.
[13]LV J,XIONG J,GUO H,et al.Joint computation offloading and resource configuration in ultra-dense edge computing networks:A deep reinforcement learning solution[C]//2019 IEEE 90th Vehicular Technology Conference(VTC2019-Fall).IEEE,2019:1-5.
[14]ZHOU H,JIANG K,LIU X,et al.Deep reinforcement learning for energy-efficient computation offloading in mobile-edge computing[J].IEEE Internet of Things Journal,2021,9(2):1517-1530.
[15]LITTLE J D C,GRAVES S C.Little's law[J].Building Intuition:Insights from Basic Operations Management Models and Principles,2008,115:81-100.
[16]HAARNOJA T,ZHOU A,ABBEEL P,et al.Soft actor-critic:Off-policy maximum entropy deep reinforcement learning with a stochastic actor[C]//International Conference on Machine Learning.PMLR,2018:1861-1870.
[17]ZENG M,HAO W,DOBRE O A,et al.Massive MIMO-assisted mobile edge computing:Exciting possibilities for computation offloading[J].IEEE Vehicular Technology Magazine,2020,15(2):31-38.
[18]LIU K H,LIAO W.Intelligent offloading for multi-access edge computing:A new actor-critic approach[C]//2020 IEEE International Conference on Communications(ICC 2020).IEEE,2020:1-6.
[1] WANG Zhihong, WANG Gaocai, ZHAO Qifei. Multi-objective Optimization of D2D Collaborative MEC Based on Improved NSGA-III [J]. Computer Science, 2024, 51(3): 280-288.
[2] WANG Xinlong, LIN Bing, CHEN Xing. Computation Offloading with Wardrop Routing Game in Multi-UAV-aided MEC Environment [J]. Computer Science, 2024, 51(3): 309-316.
[3] LIANG Chen, HONG Zheng, WU Lifa, JI Qingbing. Cryptographic Protocol Reverse Method Based on Information Entropy and Closed Frequent Sequences [J]. Computer Science, 2024, 51(3): 326-334.
[4] LI Junwei, LIU Quan, XU Yapeng. Option-Critic Algorithm Based on Mutual Information Optimization [J]. Computer Science, 2024, 51(2): 252-258.
[5] SHI Dianxi, PENG Yingxuan, YANG Huanhuan, OUYANG Qianying, ZHANG Yuhui, HAO Feng. DQN-based Multi-agent Motion Planning Method with Deep Reinforcement Learning [J]. Computer Science, 2024, 51(2): 268-277.
[6] DING Shuang, CAO Muyu, HE Xin. Online Task Offloading Decision Algorithm for High-speed Vehicles [J]. Computer Science, 2024, 51(2): 286-292.
[7] ZHAO Xiaoyan, ZHAO Bin, ZHANG Junna, YUAN Peiyan. Study on Cache-oriented Dynamic Collaborative Task Migration Technology [J]. Computer Science, 2024, 51(2): 300-310.
[8] GUO Yuxing, YAO Kaixuan, WANG Zhiqiang, WEN Liangliang, LIANG Jiye. Black-box Graph Adversarial Attacks Based on Topology and Feature Fusion [J]. Computer Science, 2024, 51(1): 355-362.
[9] LIU Xingguang, ZHOU Li, ZHANG Xiaoying, CHEN Haitao, ZHAO Haitao, WEI Jibo. Edge Intelligent Sensing Based UAV Space Trajectory Planning Method [J]. Computer Science, 2023, 50(9): 311-317.
[10] LIN Xinyu, YAO Zewei, HU Shengxi, CHEN Zheyi, CHEN Xing. Task Offloading Algorithm Based on Federated Deep Reinforcement Learning for Internet of Vehicles [J]. Computer Science, 2023, 50(9): 347-356.
[11] JIN Tiancheng, DOU Liang, ZHANG Wei, XIAO Chunyun, LIU Feng, ZHOU Aimin. OJ Exercise Recommendation Model Based on Deep Reinforcement Learning and Program Analysis [J]. Computer Science, 2023, 50(8): 58-67.
[12] XIONG Liqin, CAO Lei, CHEN Xiliang, LAI Jun. Value Factorization Method Based on State Estimation [J]. Computer Science, 2023, 50(8): 202-208.
[13] ZHANG Naixin, CHEN Xiaorui, LI An, YANG Leyao, WU Huaming. Edge Offloading Framework for D2D-MEC Networks Based on Deep Reinforcement Learningand Wireless Charging Technology [J]. Computer Science, 2023, 50(8): 233-242.
[14] CHEN Xuzhan, LIN Bing, CHEN Xing. Stackelberg Model Based Distributed Pricing and Computation Offloading in Mobile Edge Computing [J]. Computer Science, 2023, 50(7): 278-285.
[15] ZHOU Zhiqiang, ZHU Yan. Local Community Detection Algorithm for Attribute Networks Based on Multi-objective Particle Swarm Optimization [J]. Computer Science, 2023, 50(6A): 220200015-6.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.