Computer Science

Computer Science ›› 2023, Vol. 50 ›› Issue (8): 233-242.doi: 10.11896/jsjkx.220900181

Special Issue: Intelligent Edge Computing

• Computer Network • Previous Articles     Next Articles

Edge Offloading Framework for D2D-MEC Networks Based on Deep Reinforcement Learningand Wireless Charging Technology

ZHANG Naixin1, CHEN Xiaorui1, LI An1, YANG Leyao1, WU Huaming2   

  1. 1 School of Mathematics,Tianjin University,Tianjin 300192,China
    2 Center for Applied Mathematics,Tianjin University,Tianjin 300072,China
  • Received:2022-09-19 Revised:2023-02-06 Online:2023-08-15 Published:2023-08-02
  • About author:ZHANG Naixin,born in 2000,master candidate.Her main research interests include mobile edge computing and machine learning.
    WU Huaming,born in 1986,Ph.D,associate professor,Ph.D supervisor,is a member of China Computer Federation.His main research interests include Internet of things(IoT),mobile edge computing and cloud computing.
  • Supported by:
    National Natural Science Foundation of China(62071327).

PDF (PC)

448

Abstract: A large amount of underutilized computing resources in IoT devices is what mobile edge computing requires.An edge offloading framework based on device-to-device communication technology and wireless charging technology can maximize the utilization of computing resources of idle IoT devices and improve user experience.The D2D-MEC network model of IoT devices can be established on this basis.In this model,the device chooses to offload multiple tasks to multiple edge devices according to the current environment information and the estimated device state.It applies wireless charging technology to increase the success rate of transmission and computation stability.The reinforcement learning method is used to solve the joint optimization allocation problem,which aims to minimize the computation delay,energy consumption,and task dropping loss as well as maximize the utilization of edge devices and the proportion of task offloading.In addition,to adapt to larger state space and improve learning speed,an offloading scheme based on deep reinforcement learning is proposed.Based on the above theory and model,the optimal solution and upper limit of performance of the D2D-MEC system are calculated by mathematical derivation.Simulation results show that the D2D-MEC offloading model and its offloading strategy have better all-around performance and can make full use of the computing resources of IoT devices.

Key words: Mobile edge computing, Device-to-device(D2D), Reinforcement learning, Internet of things(IoT), Computation offloading, Wireless energy transmission

CLC Number: 

  • TP181
[1]BOYD B,SHILPA T,REZA A,et al.Networks and devices for the 5G era[J].IEEE Communications Magazine,2014,52(2):90-96.
[2]KO H,PACK S.Distributed Device-to-Device Offloading Sys-tem:Design and Performance Optimization[J].IEEE Transactions on Mobile Computing,2021,20(10):2949-2960.
[3]MOLIN L,CHEN T,ZENG J,et al.D2D-Assisted Computation Offloading for Mobile Edge Computing Systems with Energy Harvesting[C]//2019 20th International Conference on Parallel and Distributed Computing,Applications and Technologies (PDCAT).Gold Coast,QLD,Australia:2019:90-95.
[4]MAO Y,ZHANG J,LETAIEF K B.Dynamic computation offloading for mobile-edge computing with energy harvesting devices[J].IEEE Journal on Selected Areas in Communications,2016,34(12):3590-3605.
[5]PU L,CHEN X,XU J,et al.D2D Fogging:An Energy-Efficient and Incentive-Aware Task Offloading Framework via Network-assisted D2D Collaboration[J].IEEE Journal on Selected Areas in Communications,2016,34(12):3887-3901.
[6]FANG T,YANG Y,CHEN J X.Optimization of offloadingstrategy under assisted mobile edge computing[J].Computer Science,2022,49(S1):601-605.
[7]QIAO G H,LENG S P,ZHANG Y.Online Learning and Optimization for Computation Offloading in D2D Edge Computing and Networks[J].Mobile Networks & Applications,2019,27:1111-1122.
[8]GAIBIN L,CHEN M,WEI X,et al.Computation OffloadingWith Reinforcement Learning in D2D-MEC Network [C]//2020 International Wireless Communications and Mobile Computing (IWCMC).Limassol,Cyprus,2020:69-74.
[9]HUANG L,FENG X,QIAN L,et al.Deep ReinforcementLearning-Based Task Offloading and Resource Allocation for Mobile Edge Computing[C]//Machine Learning and Intelligent Communications(MLICOM).Cham:Springer,2018:33-42.
[10]LIANG W,WANG K,PAN C,et al.Deep Reinforcement Lear-ning Based Dynamic Trajectory Control for UAV-assisted Mobile Edge Computing[J].arXiv:1911.03887v2,2019.
[11]MINGHUI M,XIAO L,CHEN Y,et al.Learning-Based Computation Offloading for IoT Devices With Energy Harvesting[J].IEEE Transactions on Vehicular Technology,2019,68(2):1930-1941.
[12]HUANG L,BI S,ZHANG Y J A.Deep Reinforcement Learning for Online Computation Offloading in Wireless Powered Mobile-Edge Computing Networks[J].IEEE Transactions on Mobile Computing,2020,19(11):2581-2593.
[13]KAN L P G,HAUSWALD J,GAO C,et al.Collaborative Intelligence Between the Cloud and Mobile Edge[J].Acm Sigplan Notices A Monthly Publication of the Special Interest Group on Programming Languages,2017,45(1):615-629.
[14]WANG Z C,HUA J W,ZHU J Q,et al.Vehicle task offloading based on deep reinforcement learning for parking cooperation[J].Small Microcomputer System,2023,44(3):658-664.
[15]LIANG H,ZHANG L,YANG S,et al.Meta-Learning BasedDynamic Computation Task Offloading for Mobile Edge Computing Networks[J].IEEE Communications Letters,2021,25(5):1568-1572.
[16]DENG S Q,YE X G.Multi-objective task offloading algorithm based on deep Q network[J].Computer Application,2022,42(6):1668-1674.
[17]LIU P F,MAO Y C,WANG B L.Task allocation method based on cloud-fog collaboration model[J].Computer Application,2019,39(1):8-14.
[18]CHENG P,ZHANG W Z,XIE S H,et al.Research on multi-objective balanced task offloading method for edge computing of Internet of Vehicles[J].Small Microcomputer System,2022,43(9):1992-1997.
[19]DINH Q,TANG J,LA Q,et al.Offloading in Mobile Edge Computing:Task Allocation and Computational Frequency Scaling[J].IEEE Transactions on Communications,2017,65(8):3571-3584.
[20]ZHAO S,LI Y,WU D,et al.Current-Decomposition-Based Di-gital Phase Synchronization Method for BWPT System[J].IEEE Transactions on Power Electronics,2021,36(11):12183-12188.
[21]WATKINS C.Learning from delayed rewards [D].Cambridge:King's College of University of Cambridge,1989.
[22]CHRISTOPHER J,PETER D.Q-learning[J].Machine Lear-ning,1992,8(3/4):279-292.
[23]XIAO L,LI Y,HUANG X,et al.Cloud-based malware detection game for mobile devices with offloading[J].IEEE Trans Mobile Computing,2017,16(10):2742-2750.
[24]LI J,GAO H,LV T.Deep reinforcement learning based computation offloading and resource allocation for MEC[C]//IEEE Wireless Communication and Networking Conf(WCNC).2018:1-6.
[1] XIONG Liqin, CAO Lei, CHEN Xiliang, LAI Jun. Value Factorization Method Based on State Estimation [J]. Computer Science, 2023, 50(8): 202-208.
[2] XING Linquan, XIAO Yingmin, YANG Zhibin, WEI Zhengmin, ZHOU Yong, GAO Saijun. Spacecraft Rendezvous Guidance Method Based on Safe Reinforcement Learning [J]. Computer Science, 2023, 50(8): 271-279.
[3] JIN Tiancheng, DOU Liang, ZHANG Wei, XIAO Chunyun, LIU Feng, ZHOU Aimin. OJ Exercise Recommendation Model Based on Deep Reinforcement Learning and Program Analysis [J]. Computer Science, 2023, 50(8): 58-67.
[4] ZENG Qingwei, ZHANG Guomin, XING Changyou, SONG Lihua. Intelligent Attack Path Discovery Based on Hierarchical Reinforcement Learning [J]. Computer Science, 2023, 50(7): 308-316.
[5] CHEN Xuzhan, LIN Bing, CHEN Xing. Stackelberg Model Based Distributed Pricing and Computation Offloading in Mobile Edge Computing [J]. Computer Science, 2023, 50(7): 278-285.
[6] LEI Xuemei, LIU Li, WANG Qian. MEC Offloading Model Based on Linear Programming Relaxation [J]. Computer Science, 2023, 50(6A): 211200229-5.
[7] CHEN Che, ZHENG Yifeng, YANG Jingmin, YANG Liwei, ZHANG Wenjie. Dynamic Energy Optimization Strategy Based on Relay Selection and Queue Stability [J]. Computer Science, 2023, 50(6A): 220100082-8.
[8] LIN Xiangyang, XING Qinghua, XING Huaixi. Study on Intelligent Decision Making of Aerial Interception Combat of UAV Group Based onMADDPG [J]. Computer Science, 2023, 50(6A): 220700031-7.
[9] SHI Liang, WEN Liangming, LEI Sheng, LI Jianhui. Virtual Machine Consolidation Algorithm Based on Decision Tree and Improved Q-learning by Uniform Distribution [J]. Computer Science, 2023, 50(6): 36-44.
[10] WANG Hanmo, ZHENG Shijie, XU Ruonan, GUO Bin, WU Lei. Self Reconfiguration Algorithm of Modular Robot Based on Swarm Agent Deep Reinforcement Learning [J]. Computer Science, 2023, 50(6): 266-273.
[11] MIAO Kuan, LI Chongshou. Optimization Algorithms for Job Shop Scheduling Problems Based on Correction Mechanisms and Reinforcement Learning [J]. Computer Science, 2023, 50(6): 274-282.
[12] ZHANG Qiyang, CHEN Xiliang, CAO Lei, LAI Jun, SHENG Lei. Survey on Knowledge Transfer Method in Deep Reinforcement Learning [J]. Computer Science, 2023, 50(5): 201-216.
[13] YU Ze, NING Nianwen, ZHENG Yanliu, LYU Yining, LIU Fuqiang, ZHOU Yi. Review of Intelligent Traffic Signal Control Strategies Driven by Deep Reinforcement Learning [J]. Computer Science, 2023, 50(4): 159-171.
[14] XU Linling, ZHOU Yuan, HUANG Hongyun, LIU Yang. Real-time Trajectory Planning Algorithm Based on Collision Criticality and Deep Reinforcement Learning [J]. Computer Science, 2023, 50(3): 323-332.
[15] CHEN Yipeng, YANG Zhe, GU Fei, ZHAO Lei. Resource Allocation Strategy Based on Game Theory in Mobile Edge Computing [J]. Computer Science, 2023, 50(2): 32-41.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.