Computer Science

Computer Science ›› 2023, Vol. 50 ›› Issue (11A): 230200186-9.doi: 10.11896/jsjkx.230200186

• Network & Communication • Previous Articles     Next Articles

Study on Dynamic Task Offloading Scheme Based on MAB in Vehicular Edge Computing Network

XUE Jianbin, WANG Hainiu, GUAN Xiangrui, YU Bowen   

  1. Lanzhou University of Technology,Lanzhou 730050,China
  • Published:2023-11-09
  • About author:XUE Jianbin,born in 1973,Ph.D,associate professor.His main research intere-sts include wireless communication theo-ry and technology,mobile edge computing technology,etc.
    WANG Hainiu,born in 1997,postgra-duate.His main research interests include resource allocation,vehicular edge computing,and reinforcement learning.
  • Supported by:
    Gansu Science and Technology Program(23YFGA0062) and Innovation Fund of Gansu Province(2022A-215).

PDF (PC)

185

Abstract: The mobile edge computing system formed by applying mobile edge computing technology to Internet of Vehicles can provide computing services for other mobile devices through task offloading.However,due to the mobility of vehicle equipment,the environment of vehicular task offloading is dynamic and uncertain,with rapidly changing network topology,wireless channel state and computing load.These uncertainties make the task offloading process non-idealized.In view of these uncertainties,the computing resources of the MEC server are sunk into the vehicle equipment to study the task offloading between vehicles,and a solution is proposed to enable vehicles to learn the service performance of surrounding vehicles and offload tasks without knowing the status information.Based on the multi-arm bandits framework,a second-order exploration reinforcement learning algorithm is designed to maximize the average offloading return of users,and a service set update method is proposed after the end of an offloading phase to ensure the quality of service for users.Simulation results show that,compared with the existing algorithm based on upper confidence bound,the offloading return under the proposed scheme is improved by about 34%.

Key words: 6th generation mobile communication technology, Vehicular edge computing, Multi-armed bandit, Task offloading, Reinforcement learning

CLC Number: 

  • TN929.5
[1]QI W,LI Q,SONG Q,et al.Extensive edge intelligence for future vehicular networks in 6G[J].IEEE Wireless Communications,2021,28(4):128-135.
[2]KHAYYAT M,ELGENDY I A,MUTHANNAA,et al.Ad-vanced deep learning-based computational offloading for multilevel vehicular edge-cloud computing networks[J].IEEE Access,2020,8:137052-137062.
[3]KHAYYAT M,ALSHAHRANI A,ALHARBI S,et al.Multilevel service-provisioning-based autonomous vehicle applications[J].Sustainability,2020,12(6):2497.
[4]DINH T Q,TANG J,LA Q D,et al.Offloading in mobile edge computing:Task allocation and computational frequency scaling[J].IEEE Transactions on Communications,2017,65(8):3571-3584.
[5]SUN J,GU Q,ZHENGT,et al.Joint communication and computing resource allocation in vehicular edge computing[J].International Journal of Distributed Sensor Networks,2019,15(3):1550147719837859.
[6]HA K,PILLAI P,LEWIS G,et al.The impact of mobile multimedia applications on data center consolidation[C]//2013 IEEE International Conference on Cloud Engineering (IC2E).IEEE,2013:166-176.
[7]DAI Y,XU D,MAHARJAN S,et al.Joint offloading and resource allocation in vehicular edge computing and networks[C]//2018 IEEE Global Communications Conference (GLOBECOM).IEEE,2018:1-7.
[8]CHEN S,TAO Y,YU D,et al.Distributed learning dynamics of multi-armed bandits for edge intelligence[J].Journal of Systems Architecture,2021,114:101919.
[9]AUER P,CESA-BIANCHI N,FISCHER P.Finite-time analysis of the multiarmed bandit problem[J].Machine Learning,2002,47:235-256.
[10]CUI Y,LIANG Y,WANG R.Resource allocation algorithmwith multi-platform intelligent offloading in D2D-enabled vehi-cular networks[J].IEEE Access,2019,7:21246-21253.
[11]LI L,ZHOU H,XIONG S X,et al.Compound model of task arrivals and load-aware offloading for vehicular mobile edge computing networks[J].IEEE Access,2019,7:26631-26640.
[12]HAITAO Z,YI D,MENGKANG Z,et al.Multipath transmis-sion workload balancing optimization scheme based on mobile edge computing in vehicular heterogeneous network[J].IEEE Access,2019,7:116047-116055.
[13]DENG Z,CAI Z,LIANG M.A multi-hop VANETs-assisted offloading strategy in vehicular mobile edge computing[J].IEEE Access,2020,8:53062-53071.
[14]HUANG C M,LAI C F.The delay-constrained and network-situation-aware V2V2I VANET data offloading based on the multi-access edge computing(MEC) architecture[J].IEEE Open Journal of Vehicular Technology,2020,1:331-347.
[15]XUE J,HU Q,AN Y,et al.Joint task offloading and resource allocation in vehicle-assisted multi-access edge computing[J].Computer Communications,2021,177:77-85.
[16]SUN J,GU Q,ZHENG T,et al.Joint communication and computing resource allocation in vehicular edge computing[J].International Journal of Distributed Sensor Networks,2019,15(3):1550147719837859.
[17]HUANG X,YU R,XIE S,et al.Task-container matching game for computation offloading in vehicular edge computing and networks[J].IEEE Transactions on Intelligent Transportation Systems,2020,22(10):6242-6255.
[18]ZHANG Y,WANG C Y,WEI H Y.Parking reservation auction for parked vehicle assistance in vehicular fog computing[J].IEEE Transactions on Vehicular Technology,2019,68(4):3126-3139.
[19]HUANG X,YU R,YE D,et al.Efficient workload allocation and user-centric utility maximization for task scheduling in collaborative vehicular edge computing[J].IEEE Transactions on Vehicular Technology,2021,70(4):3773-3787.
[20]RAZA S,MIRZA M A,AHMAD S,et al.A vehicle to vehicle relay-based task offloading scheme in Vehicular Communication Networks[J].PeerJ Computer Science,2021,7:e486.
[21]BUDA S,GULENG S,WUC,et al.Collaborative vehicular edge computing towards greener ITS[J].IEEE Access,2020,8:63935-63944.
[22]CHA N,WU C,YOSHINAGAT,et al.Virtual edge:Exploring computation offloading in collaborative vehicular edge computing[J].IEEE Access,2021,9:37739-37751.
[23]BUTE M S,FAN P,ZHANG L,et al.An efficient distributed task offloading scheme for vehicular edge computing networks[J].IEEE Transactions on Vehicular Technology,2021,70(12):13149-13161.
[24]ZHANG X,ZHANG J,LIU Z,et al.MDP-based task offloading for vehicular edge computing under certain and uncertain transition probabilities[J].IEEE Transactions on Vehicular Techno-logy,2020,69(3):3296-3309.
[25]AUER P,CESA-BIANCHI N,FISCHER P.Finite-time analysis of the multiarmed bandit problem[J].Machine learning,2002,47(2):235-256.
[26]WU H,GUO X,LIU X.Adaptive exploration-exploitationtradeoff for opportunistic bandits[C]//International Conference on Machine Learning.PMLR,2018:5306-5314.
[27]SUN Y,GUO X,SONG J,et al.Adaptive learning-based task offloading for vehicular edge computing systems[J].IEEE Transactions on vehicular technology,2019,68(4):3061-3074.
[28]MAO Y,ZHANG J,SONG S H,et al.Stochastic joint radio and computational resource management for multi-user mobile-edge computing systems[J].IEEE Transactions on Wireless Communications,2017,16(9):5994-6009.
[29]GU X,ZHANG G,CAO Y.Cooperative mobile edge computing-cloud computing in Internet of vehicle:Architecture and energy-efficient workload allocation[J].Transactions on Emerging Telecommunications Technologies,2021,32(8):e4095.
[30]SALOMON A,AUDIBERT J Y,ALAOUI I E.Regret lowerbounds and extended Upper Confidence Bounds policies in stochastic multi-armed bandit problem[J].arXiv:1112.3827,2011.
[31]BUBECK S.Bandits games and clustering foundations[OL].Université des Sciences et Technologie de Lille-Lille I,2010.https://so1.cljtscd.com/scholar?q=Bandits+games+and+clustering+foundations.
[32]BUBECK S,CESA-BIANCHI N.Regret analysis of stochastic and nonstochastic multi-armed bandit problems[J].Foundations and Trends© in Machine Learning,2012,5(1):1-122.
[33]BNAYA Z,PUZIS R,STERNR,et al.Social network search as a volatile multi-armed bandit problem[J].Human,2013,2(2):84.
[1] LIU Xingguang, ZHOU Li, ZHANG Xiaoying, CHEN Haitao, ZHAO Haitao, WEI Jibo. Edge Intelligent Sensing Based UAV Space Trajectory Planning Method [J]. Computer Science, 2023, 50(9): 311-317.
[2] LIN Xinyu, YAO Zewei, HU Shengxi, CHEN Zheyi, CHEN Xing. Task Offloading Algorithm Based on Federated Deep Reinforcement Learning for Internet of Vehicles [J]. Computer Science, 2023, 50(9): 347-356.
[3] JIN Tiancheng, DOU Liang, ZHANG Wei, XIAO Chunyun, LIU Feng, ZHOU Aimin. OJ Exercise Recommendation Model Based on Deep Reinforcement Learning and Program Analysis [J]. Computer Science, 2023, 50(8): 58-67.
[4] XIONG Liqin, CAO Lei, CHEN Xiliang, LAI Jun. Value Factorization Method Based on State Estimation [J]. Computer Science, 2023, 50(8): 202-208.
[5] ZHANG Naixin, CHEN Xiaorui, LI An, YANG Leyao, WU Huaming. Edge Offloading Framework for D2D-MEC Networks Based on Deep Reinforcement Learningand Wireless Charging Technology [J]. Computer Science, 2023, 50(8): 233-242.
[6] XING Linquan, XIAO Yingmin, YANG Zhibin, WEI Zhengmin, ZHOU Yong, GAO Saijun. Spacecraft Rendezvous Guidance Method Based on Safe Reinforcement Learning [J]. Computer Science, 2023, 50(8): 271-279.
[7] ZENG Qingwei, ZHANG Guomin, XING Changyou, SONG Lihua. Intelligent Attack Path Discovery Based on Hierarchical Reinforcement Learning [J]. Computer Science, 2023, 50(7): 308-316.
[8] LEI Xuemei, LIU Li, WANG Qian. MEC Offloading Model Based on Linear Programming Relaxation [J]. Computer Science, 2023, 50(6A): 211200229-5.
[9] LIN Xiangyang, XING Qinghua, XING Huaixi. Study on Intelligent Decision Making of Aerial Interception Combat of UAV Group Based onMADDPG [J]. Computer Science, 2023, 50(6A): 220700031-7.
[10] WANG Hanmo, ZHENG Shijie, XU Ruonan, GUO Bin, WU Lei. Self Reconfiguration Algorithm of Modular Robot Based on Swarm Agent Deep Reinforcement Learning [J]. Computer Science, 2023, 50(6): 266-273.
[11] MIAO Kuan, LI Chongshou. Optimization Algorithms for Job Shop Scheduling Problems Based on Correction Mechanisms and Reinforcement Learning [J]. Computer Science, 2023, 50(6): 274-282.
[12] SHI Liang, WEN Liangming, LEI Sheng, LI Jianhui. Virtual Machine Consolidation Algorithm Based on Decision Tree and Improved Q-learning by Uniform Distribution [J]. Computer Science, 2023, 50(6): 36-44.
[13] ZHANG Qiyang, CHEN Xiliang, CAO Lei, LAI Jun, SHENG Lei. Survey on Knowledge Transfer Method in Deep Reinforcement Learning [J]. Computer Science, 2023, 50(5): 201-216.
[14] GAO Lixue, CHEN Xin, YIN Bo. Task Offloading Strategy Based on Game Theory in 6G Overlapping Area [J]. Computer Science, 2023, 50(5): 302-312.
[15] YU Ze, NING Nianwen, ZHENG Yanliu, LYU Yining, LIU Fuqiang, ZHOU Yi. Review of Intelligent Traffic Signal Control Strategies Driven by Deep Reinforcement Learning [J]. Computer Science, 2023, 50(4): 159-171.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.