计算机科学 ›› 2023, Vol. 50 ›› Issue (9): 347-356.doi: 10.11896/jsjkx.220800243
林欣郁, 姚泽玮, 胡晟熙, 陈哲毅, 陈星
LIN Xinyu, YAO Zewei, HU Shengxi, CHEN Zheyi, CHEN Xing
摘要： 随着车联网应用服务体系日益丰富,计算资源有限的车辆难以处理这些计算密集和时延敏感的车联网应用。计算卸载作为移动边缘计算中的一种关键技术可以解决这一难题。对于车联网中动态的多车辆多路侧单元的任务卸载环境,提出了一种基于联邦深度强化学习的任务卸载算法。该算法将每辆车都看作是智能体,采用联邦学习的框架训练各智能体,各智能体分布式决策卸载方案,以最小化系统的平均响应时间。设置评估实验,在多种动态变化的场景下对提出的算法的性能进行对比分析。实验结果显示,提出的算法求解出的系统平均响应时间短于基于规则的算法和多智能体深度强化学习算法,接近于理想方案,且求解时间远短于理想方案。实验结果表明,所提算法能够在可接受的算法执行时间内求解出接近于理想方案的系统平均响应时间。
中图分类号:
| [1]FENG J,LIU Z,WU C,et al.Mobileedge computing for the internet of vehicles:offloading framework and job scheduling[J].IEEE Vehicular Technology Magazine,2019,14(1):28-36. [2]ZHAN W,LUO C,WANG J,et al.Deep-reinforce ment-lear-ning-based offloading scheduling for vehicular edge computing[J].IEEE Internet of Things Journal,2020,7(6):5449-5465. [3]MACH P,BECVAR Z.Mobile edge computing:a survey on architecture and computation offloading[J].IEEE Communications Surveys & Tutorials,2017,19(3):1628-1656. [4]DINH H T,LEE C,NIYATO D,et al.A survey of mobile cloud computing:architecture,applications,and approaches[J].Wireless Communications and Mobile Computing,2013,13(18):1587-1611. [5]RAZA S,WANG S,AHMED M,et al.Task offloading and resource allocation for iov using 5g nr-v2x communication[J].IEEE Internet of Things Journal,2022,9(13):10397-10410. [6]LIN Y,ZHANG Y,LI J,et al.Popularity-aware online task offloading for heterogeneous vehicular edge computing using contextual clustering of bandits[J].IEEE Internet of Things Journal,2022,9(7):5422-5433. [7]DAI P,HU K,WU X,et al.A probabilistic approach for coope-rative computation offloading in mec-assisted vehicular networks[J].IEEE Transactions on Intelligent Transportation Systems,2022,23(2):899-911. [8]CHEN M,HAO Y.Task offloading for mobile edge computing in software defined ultra-dense network[J].IEEE Journal on Selected Areas in Communications,2018,36(3):587-597. [9]LIN B,GUO W,CHEN G.Scheduling strategy for scienceworkflow with deadline constraint on multi-cloud[J].Journal on Communications,2018,39(1):56-69. [10]HOU X,REN Z,WANG J,et al.Reliablecomputation offloading for edge-computing-enabled software-defined iov[J].IEEE Internet of Things Journal,2020,7(8):7097-7111. [11]LIU Z,ZHENG H,ZHANG J,et al.Computation offloading and deployment optimization in multi-uav-enabled mobile edge computing systems[J].Computer Science,2022,49(S1):619-627. [12]YAO Z,LIN J,HU J,et al.PSO-GA based approach to multi-edge load balancing[J].Computer Science,2021,48(S2):456-463. [13]ALASMARI K R,II R C G,ALAM M.Mobile edge offloading using markov decision processes[C]//Edge Computing－EDGE 2018.Seattle,WA,USA,2018. [14]SUTTON R S,BARTO A G.Reinforcement learning,secondedition:an introduction[M].Cambridge:The MIT Press,2018. [15]ZHANG K,ZHU Y,LENG S,et al.Deep learning empowered task offloading for mobile edge computing in urban informatics[J].IEEE Internet of Things Journal,2019,6(5):7635-7647. [16]MIN M,XIAO L,CHEN Y,et al.Learning-based computation offloading for iot devices with energy harvesting[J].IEEE Transactions on Vehicular Technology,2019,68(2):1930-1941. [17]HUANG L,BI S,ZHANG Y J A.Deep reinforcement learning for online computation offloading in wireless powered mobile-edge computing networks[J].IEEE Transactions on Mobile Computing,2020,19(11):2581-2593. [18]LUO Q,LI C,LUAN T H,et al.Collaborative data scheduling for vehicular edge computing via deep reinforcement learning[J].IEEE Internet of Things Journal,2020,7(10):9637-9650. [19]KE H,WANG J,DENG L,et al.Deep reinforcement learning-based adaptive computation offloading for mec in heterogeneous vehicular networks[J].IEEE Transactions on Vehicular Technology,2020,69(7):7916-7929. [20]YU S,CHEN X,ZHOU Z,et al.When deep reinforcementlearning meets federated learning:intelligent multitimescale resource management for multiaccess edge computing in 5g ultradense network[J].IEEE Internet of Things Journal,2021,8(4):2238-2251. [21]TANG M,WONG V W.Deep reinforcement learning for task offloading in mobile edge computing systems[J].IEEE Transactions on Mobile Computing,2022,21(6):1985-1997. [22]HUANG X,LENG S,MAHARJAN S,et al.Multi-agent deep reinforcement learning for computation offloading and interfe-rence coordination in small cell networks[J].IEEE Transactions on Vehicular Technology,2021,70(9):9282-9293. [23]CHEN X,HU J,CHEN Z,et al.A reinforcement learning-empowered feedback control system for industrial internet of things[J].IEEE Transactions on Industrial Informatics,2022,18(4):2724-2733. [24]MNIH V,KAVUKCUOGLU K,SILVER D,et al.Human-level control through deep reinforcement learning[J].Nature,2015,518(7540):529-533. [25]JIA M,LIANG W,XU Z,et al.Qos-aware cloudlet load balancing in wireless metropolitan area networks[J].IEEE Transactions on Cloud Computing,2020,8(2):623-634. [26]KLEINROCK L.Queueing systems,volume I:theory[M].Hoboken:Wiley,1975:101-103. [27]LIANG L,YE H,LI G Y.Spectrum sharing in vehicular networks based on multi-agent reinforcement learning[J].IEEE Journal on Selected Areas in Communications,2019,37(10):2282-2292. [28]WATKINS C J C H.Learning from delayed rewards[D].Cambridge:University of Cambridge,1989. [29]HINTON G.Overview of mini-batch gradient descent[EB/OL].http://www.cs.toronto.edu/~tijmen/csc321/slides/lecture_slides_lec6.pdf. [30]WU Y,GUO K,HUANG J,et al.Secrecy-based energy-efficient data offloading via dual connectivity over unlicensed spectrums[J].IEEE Journal on Selected Areas in Communications,2016,34(12):3252-3270. [31]NAIR V,HINTON G E.Rectified linear units improve restric-ted boltzmann machines[C]//Proceedings of the 27th International Conference on Machine Learning(ICML-10).Haifa,Is-rael,2010:807-814. |
|
||
首页