Computer Science

Computer Science ›› 2022, Vol. 49 ›› Issue (7): 248-253.doi: 10.11896/jsjkx.210400219

• Computer Network • Previous Articles     Next Articles

Edge-Cloud Collaborative Resource Allocation Algorithm Based on Deep Reinforcement Learning

YU Bin1, LI Xue-hua1, PAN Chun-yu1, LI Na2   

  1. 1 School of Information and Telecommunication Engineering,Beijing Information Science & Technology University,Beijing 100192,China
    2 Baicells Joint Laboratory of Intelligent and IoT,Beijing Information Science and Technology University,Beijing 100094,China
  • Received:2021-04-21 Revised:2022-03-08 Online:2022-07-15 Published:2022-07-12
  • About author:YU Bin,born in 1996,postgraduate.His main research interests include mobile edge computing and resource allocation.
    LI Xue-hua,born in 1977,Ph.D,professor.Her main research interests include wireless communication technologies,internet of things technologies and smart edge computing.
  • Supported by:
    Natural Science Foundation of Beijing with Municipal Education Commission Joint Fund(KZ201911232046) and Natural Science Foundation of Beijing with Haidian Original Innovation Joint Fund(L192022,L182039).

PDF (PC)

1005

Abstract: Mobile Edge Computing(MEC) is used to enhance data processing in low power networks,and it has become an efficient computing paradigm.This paper considers an edge-cloud collaborative system composed of multiple MTs and adopts a variety of offloading modes.In order to reduce the total time delay of MTs,a task offloading algorithm based on deep reinforcement learning is proposed.It implements deep neural network(DNN) as a scalable solution,learns the multi-base offloading mode from experience to minimize the total time delay.Simulation results indicate that compared with the deep Q network(DQN) algorithm and the deep deterministic policy gradient(DDPG) algorithm,the proposed algorithm can improve the maximum performance gain significantly.In addition,the proposed algorithm has good convergence,and its result can approach the optimal result obtained by exhaustive search.

Key words: Deep reinforcement learning, Mobile edge computing, Mobile terminal, Resource allocation

CLC Number: 

  • TP181
[1]WANG K Z,YANG K,MAGURAWALAGE C S.Joint Energy Minimization and Resource Allocation in C-RAN with Mobile Cloud[J].IEEE Transactions on Cloud Computing,2018,6(3):760-770.
[2]WANG S,ZHANG X,ZHANG Y,et al.A Survey on MobileEdge Networks:Convergence of Computing,Caching and Communications[J].IEEE Access,2017,5:6757-6779.
[3]ABBAS N,ZHANG Y,TAHERKORDI A,et al.Mobile Edge Computing:A Survey[J].IEEE Internet of Things Journal,2017,5(1):450-465.
[4]WANG S H,PAN C Y,YIN C C.Joint Heterogeneous Tasks Offloading and Resource Allocation in Mobile Edge Computing Systems[C]//2018 10th International Conference on Wireless Communications and Signal Processing(WCSP).2018.
[5]ZHANG J,WU M Q,ZHAO M.Joint Computation Offloading and Resource Allocation in C-RAN With MEC Based on Spectrum Efficiency[J].IEEE Access,2019,7:79056-79068.
[6]YU W J,MUSAVIAN L,QUDDUS A U,et al.Low LatencyDriven Effective Capacity Analysis for Non-Orthogonal and Orthogonal Spectrum Access[C]//2018 IEEE Globecom Workshops(GC Wkshps).IEEE,2019.
[7]WU Y,NI K,ZHANG C,et al.NOMA Assisted Multi-Access Mobile Edge Computing:A Joint Optimization of Computation Offloading and Time Allocation[J].IEEE Transactions on Vehicular Technology,2018,67(12):12244-12258.
[8]ZHAI D S,ZHANG R N,CAI L,et al.Energy-efficient user scheduling and power allocation for NOMA-based wireless networks with massive IoT devices[J].IEEE Internet of Things Journal,2018,5(3):1857-1868.
[9]LIU Q,ZHAI J W,ZHANG Z C,et al.A Survey on Deep Reinforcement Learning[J].Chinese Journal of Computers,2018,41(1):1-27.
[10]LIANG J B,ZHANG H H,JIANG C,et al.Research Progress of Task Offloading Based on Deep Reinforcement Learning in Mobile Edge Computing [J].Computer Science,2021,48(7):316-323.
[11]YANG Z,LIU Y W,CHEN Y,et al.Deep Reinforcement Lear-ning in Cache-Aided MEC Networks[C]//IEEE International Conference on Communications.2019,3:20-24.
[12]HE X M,WANG K,HUANG H W,et al.Green resource allocation based on deep reinforcement learning in content-centric IoT[J].IEEE Transactions on Emerging Topics in Computing,2018,10(3):1-16.
[13]LIU N,LI Z,XU J L,et al.A hierarchical framework of cloud resource allocation and power management using deep reinforcement learning[C]//IEEE 37th International Conference on Distributed Computing Systems(ICDCS).IEEE,2017:372-382.
[14]CHEN Z,WANG X D.Decentralized Computation Offloading for Multi-User Mobile Edge Computing:A Deep Reinforcement Learning Approach[J].arXiv:1812.07394,2018.
[15]ALFAKIH T,HASSAN M M,GUMAEI A,et al.Task Off-loading and Resource Allocation for Mobile Edge Computing by Deep Reinforcement Learning Based on SARSA[J].IEEE Access,2020,8:54074-54084.
[16]LI J,GAO H,LV T J,et al.Deep Reinforcement Learning based Computation Offloading and Resource Allocation for MEC[J].IEEE Wireless Communications and Networking Conference,2018:1-6.
[17]TANG M,WONG W S V.Deep Reinforcement Learning Based Task Offloading Algorithm for Mobile-edge Computing Systems[J].arXiv:2005.02459,2020.
[18]YAN J,BI S Z,ZHANG A J Y.Offloading and Resource Allocation With General Task Graph in Mobile Edge Computing:A Deep Reinforcement Learning Approach[J].IEEE Transactions on Wireless Communications,2020,19(8):5404-5419.
[19]CANEDO J,SKJELLUM A.Using machine learning to secure IoT systems[C]//2016 14th Annual Conference on Privacy,Security and Trust(PST).IEEE,2016:219-222.
[20]TORRES P,CATANIA C,GARCIA S,et al.An analysis of Recurrent Neural Networks for Botnet detection behavior[C]//2016 IEEE Biennial Congress of Argentina(ARGENCON).IEEE,2016:1-6.
[21]TIAN B,ZHANG Q,XIN X J,et al.Recursive Neural Network Based RRH to BBU Resource Allocation in 5G Fronthaul Network[C]//2018 Asia Communications and Photonics Confe-rence(ACP).IEEE,2018:1-3.
[22]FENG J,YU F R,PEI Q Q,et al.Cooperative Computation Offloading and Resource Allocation for Blockchain-Enabled Mobile-Edge Computing:A Deep Reinforcement Learning Approach[J].IEEE Internet of Things Journal,2020,7(7):6214-6228.
[23]WANG D,QIN H,SONG B,et al.Resource Allocation in Information-Centric Wireless Networking with D2D-Enabled MEC:A Deep Reinforcement Learning Approach[J].IEEE Access,2019,7:114935-114944.
[24]HUANG B B,LI Z J,XU Y Q,et al.Deep Reinforcement Lear-ning for Performance-Aware Adaptive Resource Allocation in Mobile Edge Computing[J].Wireless Communications and Mobile Computing,2020(2020):1-17.
[25]XIONG X,ZHENG K,LEI L,et al.Resource Allocation Based on Deep Reinforcement Learning in IoT Edge Computing[J].2020 IEEE Journal on Selected Areas in Communications,2020,38(6):1133-1146.
[26]FAN Y F,YUAN S,CAI Y,et al.Deep Reinforcement Lear-ning-based Collaborative Computation Offloading Scheme in Vehicular Edge Computing[J].Computer Science,2021,48(5):270-276.
[27]MA Y Y,ZHENG W B,MA Y,et al.Multi-workflow Offloading Method Based on Deep Reinforcement Learning and Probabilistic Performance-aware in Edge Computing Environment [J].Computer Science,2021,48(1):40-48.
[28]YANG T,ZHANG H L,JI H,et al.Computation collaboration in ultra dense network integrated with mobile edge computing[C]//IEEE 28th Annual International Symposium on Personal,Indoor,and Mobile Radio Communications.2017:1-5.
[29]3GPP TR 36.885 V2.0.0-2016,3rd Generation PartnershipProject:Technical Specification Group Radio Access Network:Study LTE-Based V2X Services(Release 14) [S]:2016.
[1] TANG Feng, FENG Xiang, YU Hui-qun. Multi-task Cooperative Optimization Algorithm Based on Adaptive Knowledge Transfer andResource Allocation [J]. Computer Science, 2022, 49(7): 254-262.
[2] LI Meng-fei, MAO Ying-chi, TU Zi-jian, WANG Xuan, XU Shu-fang. Server-reliability Task Offloading Strategy Based on Deep Deterministic Policy Gradient [J]. Computer Science, 2022, 49(7): 271-279.
[3] YAO Ye, ZHU Yi-an, QIAN Liang, JIA Yao, ZHANG Li-xiang, LIU Rui-liang. Android Malware Detection Method Based on Heterogeneous Model Fusion [J]. Computer Science, 2022, 49(6A): 508-515.
[4] FANG Tao, YANG Yang, CHEN Jia-xin. Optimization of Offloading Decisions in D2D-assisted MEC Networks [J]. Computer Science, 2022, 49(6A): 601-605.
[5] LIU Zhang-hui, ZHENG Hong-qiang, ZHANG Jian-shan, CHEN Zhe-yi. Computation Offloading and Deployment Optimization in Multi-UAV-Enabled Mobile Edge Computing Systems [J]. Computer Science, 2022, 49(6A): 619-627.
[6] XIE Wan-cheng, LI Bin, DAI Yue-yue. PPO Based Task Offloading Scheme in Aerial Reconfigurable Intelligent Surface-assisted Edge Computing [J]. Computer Science, 2022, 49(6): 3-11.
[7] ZHOU Tian-qing, YUE Ya-li. Multi-Task and Multi-Step Computation Offloading in Ultra-dense IoT Networks [J]. Computer Science, 2022, 49(6): 12-18.
[8] QIU Xu, BIAN Hao-bu, WU Ming-xiao, ZHU Xiao-rong. Study on Task Offloading Algorithm for Internet of Vehicles on Highway Based on 5G MillimeterWave Communication [J]. Computer Science, 2022, 49(6): 25-31.
[9] XU Hao, CAO Gui-jun, YAN Lu, LI Ke, WANG Zhen-hong. Wireless Resource Allocation Algorithm with High Reliability and Low Delay for Railway Container [J]. Computer Science, 2022, 49(6): 39-43.
[10] HONG Zhi-li, LAI Jun, CAO Lei, CHEN Xi-liang, XU Zhi-xiong. Study on Intelligent Recommendation Method of Dueling Network Reinforcement Learning Based on Regret Exploration [J]. Computer Science, 2022, 49(6): 149-157.
[11] SHEN Jia-fang, QIAN Li-ping, YANG Chao. Non-orthogonal Multiple Access and Multi-dimension Resource Optimization in EH Relay NB-IoT Networks [J]. Computer Science, 2022, 49(5): 279-286.
[12] LI Peng, YI Xiu-wen, QI De-kang, DUAN Zhe-wen, LI Tian-rui. Heating Strategy Optimization Method Based on Deep Learning [J]. Computer Science, 2022, 49(4): 263-268.
[13] OUYANG Zhuo, ZHOU Si-yuan, LYU Yong, TAN Guo-ping, ZHANG Yue, XIANG Liang-liang. DRL-based Vehicle Control Strategy for Signal-free Intersections [J]. Computer Science, 2022, 49(3): 46-51.
[14] ZHANG Hai-bo, ZHANG Yi-feng, LIU Kai-jian. Task Offloading,Migration and Caching Strategy in Internet of Vehicles Based on NOMA-MEC [J]. Computer Science, 2022, 49(2): 304-311.
[15] PAN Yan-na, FENG Xiang, YU Hui-qun. Competitive-Cooperative Coevolution for Large Scale Optimization with Computation Resource Allocation Pool [J]. Computer Science, 2022, 49(2): 182-190.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.