计算机科学 ›› 2021, Vol. 48 ›› Issue (5): 270-276.doi: 10.11896/jsjkx.201000005

• 计算机网络 • 上一篇    下一篇

车载边缘计算中基于深度强化学习的协同计算卸载方案

范艳芳, 袁爽, 蔡英, 陈若愚   

  1. 北京信息科技大学计算机学院 北京100101
  • 收稿日期:2020-10-03 修回日期:2021-01-05 出版日期:2021-05-15 发布日期:2021-05-09
  • 通讯作者: 范艳芳(fyfhappy@bistu.edu.cn)
  • 基金资助:
    国家自然科学基金(61672106);北京市自然科学基金(L192023);北京信息科技大学基金(2025028);北京信息科技大学研究生科技创新项目、网络文化与数字传播北京市重点实验室开放课题

Deep Reinforcement Learning-based Collaborative Computation Offloading Scheme in VehicularEdge Computing

FAN Yan-fang, YUAN Shuang, CAI Ying, CHEN Ruo-yu   

  1. School of Computer,Beijing Information Science & Technology University,Beijing 100101,China
  • Received:2020-10-03 Revised:2021-01-05 Online:2021-05-15 Published:2021-05-09
  • About author:FAN Yan-fang,born in 1979,Ph.D,is a member of China Computer Federation.Her main research interests include information security,vehicular networking and edge computing.
  • Supported by:
    National Natural Science Foundation of China(61672106),Natural Science Foundation of Beijing(L192023),Foundation of Beijing Information Science & Technology University(2025028),Graduate Science and Technology Innovation Project of Beijing Information Science & Technology University and Open Project of Beijing Key Laboratory of Internet Culture and Digital Dissemination Research.

摘要: 车载边缘计算(Vehicular Edge Computing,VEC)是一种可实现车联网低时延和高可靠性的关键技术,用户将计算任务卸载到移动边缘计算(Mobile Edge Computing,MEC)服务器上,不仅可以解决车载终端计算能力不足的问题,而且可以减少能耗,降低车联网通信服务的时延。然而,高速公路场景下车辆移动性与边缘服务器静态部署的矛盾给计算卸载的可靠性带来了挑战。针对高速公路环境的特点,研究了临近车辆提供计算服务的可能性。通过联合MEC服务器和车辆的计算资源,设计并实现了一个基于深度强化学习的协同计算卸载方案,以实现在满足任务时延约束的前提下最小化所有任务时延的目标。仿真实验结果表明,相比于没有车辆协同的方案,所提方案可以有效降低时延和计算卸载失败率。

关键词: 车载边缘计算, 计算卸载, 深度强化学习, 协同计算, 移动边缘计算

Abstract: Vehicular edge computing (VEC) is a key technology that can realize low latency and high reliability of internet of vehicles.Users offload computing tasks to mobile edge computing (MEC) servers,which can not only solve the problem of insufficient computing capability of vehicles,but also reduce the energy consumption and the latency of communication service.How-ever,the contradiction between the mobility of vehicles and the static deployment of edge servers in highway scenarios poses a challenge to the reliability of computing offloading.To solve this problem,this paper designs a collaborative deep reinforcement learning-based scheme for vehicles to adapt to the dynamic high-speed environment by combining the computing resources of MEC servers and neighboring vehicles.Simulation results show that compared with the scheme without vehicle collaboration,this scheme can reduce the delay and the failure rate of offloading.

Key words: Collaborative computing, Computation offloading, Deep reinforcement learning, Mobile edge computing, Vehicular edge computing

中图分类号: 

  • TN929.5
[1]ETSI.MEC in 5G networks[OL]. https://www.etsi.org/images/files/ETSIWhitePapers/etsi_wp28_mec_in_5G_FINAL.pdf.
[2]YUAN S,FAN Y,CAI Y.A survey on computation offloading for vehicular edge computing[C]// Proceedings of the 2019 7th International Conference on Information Technology:IoT and Smart City (ICIT 2019).New York:ACM Press,2019:107-112.
[3]YU P,ZHANG J,LI W,et al.Energy-efficient resource allocation method in mobile edge network based on double deep Q-learning [J].Journal on Communications,2020,41(12):148-161.
[4]MEHDI M,ALA A F,SAMEH S,et al.Deep Learning for IoT Big Data and Streaming Analytics:A Survey [J].IEEE Communications Surveys & Tutorials,2018,20(4):2923-296.
[5]NING Z L,FENG Y F,COLLOTTA M,et al.Deep learning in edge of vehicles:Exploring relationship for data transmission [J].IEEE Transactions on Industrial Informatics,2019,15(10):5737-5746.
[6]ZHANG J,GUO H,LIU H,et al.Task offloading in vehicular edge computing networks:A load-balancing solution [J].IEEE Transactions on Vehicular Technology,2020,69(2):2092-2104.
[7]KHAN I,TAO X,RAHMAN G M S,et al.Advanced Energy-Efficient Computation Offloading Using Deep Reinforcement Learning in MTC Edge Computing [J].IEEE Access,2020,8:82867-82875.
[8]HUANG X,YU R,KANG J,et al.Exploring mobile edge computing for 5g-enabled software defined vehicular networks [J].IEEE Wireless Commun,2017,24(6):55-63.
[9]QIAO G,LENG S,ZHANG K,et al.Collaborative Task Off-loading in Vehicular Edge Multi-Access Networks [J].IEEE Communications Magazine,2018,56(8):48-54.
[10]DAI S,WANG H L,GAO Z,et al.An adaptive computation offloading mechanism for mobile health applications [J].IEEE Transactions on Vehicular Technology,2020,69(1):998-1007.
[11]HUANG X,YU R,LIU J,et al.Parked vehicle edge computing:Exploiting opportunistic resources for distributed mobile applications [J].IEEE Access,2018,6:66649-66663.
[12]LI Y,YANG B,CHEN Z,et al.A Contract-Stackelberg Offloa-ding Incentive Mechanism for Vehicular Parked-Edge Computing Networks[C]//Proceedings of 2019 IEEE 89th Vehicular Technology Conference (VTC2019-Spring).Piscataway:IEEE Press,2019:1-5.
[13]LIU J,WANG S,WANG J,et al.A task-oriented computationoffloading algorithm for intelligent vehicle network with mobile edge computing [J].IEEE Access,2019,7:180491-180502.
[14]NING Z,DONG P,WANG X,et al.When Deep ReinforcementLearning Meets 5G Vehicular Networks:A Distributed Offloading Framework for Traffic Big Data [J].IEEE Transactions on Industrial Informatics,2019,16(2):1352-1361.
[1] 熊丽琴, 曹雷, 赖俊, 陈希亮.
基于值分解的多智能体深度强化学习综述
Overview of Multi-agent Deep Reinforcement Learning Based on Value Factorization
计算机科学, 2022, 49(9): 172-182. https://doi.org/10.11896/jsjkx.210800112
[2] 孙慧婷, 范艳芳, 马孟晓, 陈若愚, 蔡英.
VEC中基于动态定价的车辆协同计算卸载方案
Dynamic Pricing-based Vehicle Collaborative Computation Offloading Scheme in VEC
计算机科学, 2022, 49(9): 242-248. https://doi.org/10.11896/jsjkx.210700166
[3] 于滨, 李学华, 潘春雨, 李娜.
基于深度强化学习的边云协同资源分配算法
Edge-Cloud Collaborative Resource Allocation Algorithm Based on Deep Reinforcement Learning
计算机科学, 2022, 49(7): 248-253. https://doi.org/10.11896/jsjkx.210400219
[4] 张翀宇, 陈彦明, 李炜.
边缘计算中面向数据流的实时任务调度算法
Task Offloading Online Algorithm for Data Stream Edge Computing
计算机科学, 2022, 49(7): 263-270. https://doi.org/10.11896/jsjkx.210300195
[5] 李梦菲, 毛莺池, 屠子健, 王瑄, 徐淑芳.
基于深度确定性策略梯度的服务器可靠性任务卸载策略
Server-reliability Task Offloading Strategy Based on Deep Deterministic Policy Gradient
计算机科学, 2022, 49(7): 271-279. https://doi.org/10.11896/jsjkx.210600040
[6] 方韬, 杨旸, 陈佳馨.
D2D辅助移动边缘计算下的卸载策略优化
Optimization of Offloading Decisions in D2D-assisted MEC Networks
计算机科学, 2022, 49(6A): 601-605. https://doi.org/10.11896/jsjkx.210200114
[7] 刘漳辉, 郑鸿强, 张建山, 陈哲毅.
多无人机使能移动边缘计算系统中的计算卸载与部署优化
Computation Offloading and Deployment Optimization in Multi-UAV-Enabled Mobile Edge Computing Systems
计算机科学, 2022, 49(6A): 619-627. https://doi.org/10.11896/jsjkx.210600165
[8] 谢万城, 李斌, 代玥玥.
空中智能反射面辅助边缘计算中基于PPO的任务卸载方案
PPO Based Task Offloading Scheme in Aerial Reconfigurable Intelligent Surface-assisted Edge Computing
计算机科学, 2022, 49(6): 3-11. https://doi.org/10.11896/jsjkx.220100249
[9] 周天清, 岳亚莉.
超密集物联网络中多任务多步计算卸载算法研究
Multi-Task and Multi-Step Computation Offloading in Ultra-dense IoT Networks
计算机科学, 2022, 49(6): 12-18. https://doi.org/10.11896/jsjkx.211200147
[10] 洪志理, 赖俊, 曹雷, 陈希亮, 徐志雄.
基于遗憾探索的竞争网络强化学习智能推荐方法研究
Study on Intelligent Recommendation Method of Dueling Network Reinforcement Learning Based on Regret Exploration
计算机科学, 2022, 49(6): 149-157. https://doi.org/10.11896/jsjkx.210600226
[11] 李鹏, 易修文, 齐德康, 段哲文, 李天瑞.
一种基于深度学习的供热策略优化方法
Heating Strategy Optimization Method Based on Deep Learning
计算机科学, 2022, 49(4): 263-268. https://doi.org/10.11896/jsjkx.210300155
[12] 彭冬阳, 王睿, 胡谷雨, 祖家琛, 王田丰.
视频缓存策略中QoE和能量效率的公平联合优化
Fair Joint Optimization of QoE and Energy Efficiency in Caching Strategy for Videos
计算机科学, 2022, 49(4): 312-320. https://doi.org/10.11896/jsjkx.210800027
[13] 欧阳卓, 周思源, 吕勇, 谭国平, 张悦, 项亮亮.
基于深度强化学习的无信号灯交叉路口车辆控制
DRL-based Vehicle Control Strategy for Signal-free Intersections
计算机科学, 2022, 49(3): 46-51. https://doi.org/10.11896/jsjkx.210700010
[14] 张海波, 张益峰, 刘开健.
基于NOMA-MEC的车联网任务卸载、迁移与缓存策略
Task Offloading,Migration and Caching Strategy in Internet of Vehicles Based on NOMA-MEC
计算机科学, 2022, 49(2): 304-311. https://doi.org/10.11896/jsjkx.210100157
[15] 代珊珊, 刘全.
基于动作约束深度强化学习的安全自动驾驶方法
Action Constrained Deep Reinforcement Learning Based Safe Automatic Driving Method
计算机科学, 2021, 48(9): 235-243. https://doi.org/10.11896/jsjkx.201000084
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!