计算机科学 ›› 2022, Vol. 49 ›› Issue (6): 3-11.doi: 10.11896/jsjkx.220100249
谢万城1, 李斌1,2, 代玥玥3
XIE Wan-cheng1, LI Bin1,2, DAI Yue-yue3
摘要： 针对6G时代“智慧物联网”边缘计算系统中障碍物阻挡对任务卸载性能的影响,提出了一种无人机搭载智能反射面(Reconfigurable Intelligent Surfaces,RIS)辅助的计算任务部分卸载方案。首先,在满足用户传输功率、无人机高度、任务卸载比例限制的条件下,通过联合优化时隙分配、任务卸载比例、无人机高度、RIS相移和用户传输功率,建立用户总能耗最小化问题;其次,将该非凸优化问题分解为4个子问题,使用深度强化学习中的近端策略优化(Proximal Policy Optimization,PPO)方法确定时隙分配策略;最后,将每个训练时间步作为一次求解,基于交替迭代方法和连续凸逼近方法得到问题的优化解。仿真结果表明,基于PPO的算法训练速度较快其用户总能耗比采用全部卸载方案的能耗减少了约23%,比采用无人机高度固定方案的能耗减少了约5.3%。
中图分类号:
| [1] WU D P,ZHANG P N,WANG R Y.Smart Internet of things Aided by “Terminal-Edge-Cloud” Cooperation[J].Chinese Journal on Internet of Things,2018,2(3):21-28. [2] LI Z J,ZHANG X L.Resource Allocation and Offloading Decision of Edge Computing for Reducing Core Network Congestion[J].Computer Science,2021,48(3):281-288. [3] XU Y,ZHANG T,YANG D,et al.UAV-Assisted Relaying and MEC Networks:Resource Allocation and 3D Deployment[C]//2021 IEEE International Conference on Communications Workshops (ICC Workshops).2021:1-6. [4] ZHANG T,XU Y,LOO J,et al.Joint Computation and Communication Design for UAV-Assisted Mobile Edge Computing in IoT[J].IEEE Transactions on Industrial Informatics,2020,16(8):5505-5516. [5] DIAO X B,YANG W D,YANG L X,et al.UAV-Relaying-Assisted Multi-Access Edge Computing With Multi-Antenna Base Station:Offloading and Scheduling Optimization[J].IEEE Transactions on Vehicular Technology,2021,70(9):9495-9509. [6] WANG J,NA Z,LIU X.Collaborative Design of Multi-UAVTrajectory and Resource Scheduling for 6G-Enabled Internet of Things[J].IEEE Internet of Things Journal,2021,8(20):15096-15106. [7] TIAN H,NI W L,WANG W,et al.Data-Importance-Aware Resource Allocation in IRS-Aided Edge Intelligent System[J].Journal of Beijing University of Posts and Telecommunications,2020,43(6):51-58. [8] LI Z Y,CHEN M,YANG Z H,et al.Energy Efficient Reconfi-gurable Intelligent Surface Enabled Mobile Edge Computing Networks With NOMA[J].IEEE Transactions on Cognitive Communications and Networking,2021,7(2):427-440. [9] LI A C,LIU Y,LI M,et al.Joint Scheduling Design in Wireless Powered MEC IoT Networks Aided by Reconfigurable Intelligent Surface[C]//2021 IEEE/CIC International Conference on Communications in China (ICCC Workshops).Xiamen,China:IEEE,2021:159-164. [10] HUANG S F,WANG S,WANG R,et al.Reconfigurable Intelligent Surface Assisted Mobile Edge Computing With Heterogeneous Learning Tasks[J].IEEE Transactions on Cognitive Communications and Networking,2021,7(2):369-382. [11] YANG Z H,HUANG C W,SHI J F,et al.Optimal Control for Full-Duplex Communications with Reconfigurable Intelligent Surface[C]//ICC 2021－IEEE International Conference on Communications.Montreal,QC,Canada:IEEE,2021:1-6. [12] LIU X,LIU Y W,CHEN Y.Machine Learning EmpoweredTrajectory and Passive Beamforming Design in UAV-RIS Wireless Networks[J].IEEE Journal on Selected Areas in Communications,2021,39(7):2042-2055. [13] MEI H B,YANG K,SHEN J,et al.Joint Trajectory-Task-Cache Optimization With Phase-Shift Design of RIS-Assisted UAV for MEC[J].IEEE Wireless Communications Letters,2021,10(7):1586-1590. [14] LONG H,CHEN M,YANG Z H,et al.Joint Trajectory andPassive Beamforming Design for Secure UAV Networks with RIS[C]//2020 IEEE Globecom Workshops.Taipei,Taiwan:IEEE,2020:1-6. [15] MURSIA P,DEVOTI F,SCIANCALEPORE V,et al.RISe ofFlight:RIS-Empowered UAV Communications for Robust and Reliable Air-to-Ground Networks[J].IEEE Open Journal of the Communications Society,2021,2:1616-1629. [16] SAMIR M,ELHATTAB M,ASSI C,et al.Optimizing Age of Information Through Aerial Reconfigurable Intelligent Surfaces:A Deep Reinforcement Learning Approach[J].IEEE Transactions on Vehicular Technology,2021,70(4):3978-3983. [17] ZHAN C,HU H,SUI X F,et al.Completion Time and Energy Optimization in the UAV-Enabled Mobile-Edge Computing System[J].IEEE Internet of Things Journal,2020,7(8):7808-7822. [18] LI A,DAI L B,YU L S,et al.Resource Allocation for Un-manned Aerial Vehicle-assisted Mobile Edge Computing to Mini-mize Weighted Energy Consumption[J].Journal of Electronics &Information Technology,2021:1-8. [19] WANG F,XU J,CUI S.Optimal Energy Allocation and TaskOffloading Policy for Wireless Powered Mobile Edge Computing Systems[J].IEEE Transactions on Wireless Communications,2020,19(4):2443-2459. [20] WANG F,XU J,WANG X,et al.Joint Offloading and Computing Optimization in Wireless Powered Mobile-Edge Computing Systems[J].IEEE Transactions on Wireless Communications,2018,17(3):1784-1797. [21] LIANG J B,ZHANG H H,JIANG C,et al.Research Progress of Task Offloading Based on Deep Reinforcement Learning in Mobile Edge Computing[J].Computer Science,2021,48(7):316-323. [22] ENGSTROM L,ILYAS A,SANTURKAR S,et al.Implementa-tion Matters in Deep Policy Gradients:A Case Study on PPO and TRPO[C]//2020 International Conference on Learning Representations.2019:1-14. [23] LIU C H,DAI Z,ZHAO Y,et al.Distributed and Energy-Efficient Mobile Crowdsensing with Charging Stations by Deep Reinforcement Learning[J].IEEE Transactions on Mobile Computing,2021,20(1):130-146. [24] BOYD S,VANDENBERGHE L.Convex Optimization[M].Cambridge:Cambridge University Press,2004. |
| [1] | 熊丽琴, 曹雷, 赖俊, 陈希亮. 基于值分解的多智能体深度强化学习综述 Overview of Multi-agent Deep Reinforcement Learning Based on Value Factorization 计算机科学, 2022, 49(9): 172-182. https://doi.org/10.11896/jsjkx.210800112 |
| [2] | 蹇奇芮, 陈泽茂, 武晓康. 面向无人机通信的认证和密钥协商协议 Authentication and Key Agreement Protocol for UAV Communication 计算机科学, 2022, 49(8): 306-313. https://doi.org/10.11896/jsjkx.220200098 |
| [3] | 于滨, 李学华, 潘春雨, 李娜. 基于深度强化学习的边云协同资源分配算法 Edge-Cloud Collaborative Resource Allocation Algorithm Based on Deep Reinforcement Learning 计算机科学, 2022, 49(7): 248-253. https://doi.org/10.11896/jsjkx.210400219 |
| [4] | 李梦菲, 毛莺池, 屠子健, 王瑄, 徐淑芳. 基于深度确定性策略梯度的服务器可靠性任务卸载策略 Server-reliability Task Offloading Strategy Based on Deep Deterministic Policy Gradient 计算机科学, 2022, 49(7): 271-279. https://doi.org/10.11896/jsjkx.210600040 |
| [5] | 方韬, 杨旸, 陈佳馨. D2D辅助移动边缘计算下的卸载策略优化 Optimization of Offloading Decisions in D2D-assisted MEC Networks 计算机科学, 2022, 49(6A): 601-605. https://doi.org/10.11896/jsjkx.210200114 |
| [6] | 刘漳辉, 郑鸿强, 张建山, 陈哲毅. 多无人机使能移动边缘计算系统中的计算卸载与部署优化 Computation Offloading and Deployment Optimization in Multi-UAV-Enabled Mobile Edge Computing Systems 计算机科学, 2022, 49(6A): 619-627. https://doi.org/10.11896/jsjkx.210600165 |
| [7] | 陈博琛, 唐文兵, 黄鸿云, 丁佐华. 基于改进人工势场的未知障碍物无人机编队避障 Pop-up Obstacles Avoidance for UAV Formation Based on Improved Artificial Potential Field 计算机科学, 2022, 49(6A): 686-693. https://doi.org/10.11896/jsjkx.210500194 |
| [8] | 周天清, 岳亚莉. 超密集物联网络中多任务多步计算卸载算法研究 Multi-Task and Multi-Step Computation Offloading in Ultra-dense IoT Networks 计算机科学, 2022, 49(6): 12-18. https://doi.org/10.11896/jsjkx.211200147 |
| [9] | 邱旭, 卞浩卜, 吴铭骁, 朱晓荣. 基于5G毫米波通信的高速公路车联网任务卸载算法研究 Study on Task Offloading Algorithm for Internet of Vehicles on Highway Based on 5G MillimeterWave Communication 计算机科学, 2022, 49(6): 25-31. https://doi.org/10.11896/jsjkx.211100198 |
| [10] | 洪志理, 赖俊, 曹雷, 陈希亮, 徐志雄. 基于遗憾探索的竞争网络强化学习智能推荐方法研究 Study on Intelligent Recommendation Method of Dueling Network Reinforcement Learning Based on Regret Exploration 计算机科学, 2022, 49(6): 149-157. https://doi.org/10.11896/jsjkx.210600226 |
| [11] | 李鹏, 易修文, 齐德康, 段哲文, 李天瑞. 一种基于深度学习的供热策略优化方法 Heating Strategy Optimization Method Based on Deep Learning 计算机科学, 2022, 49(4): 263-268. https://doi.org/10.11896/jsjkx.210300155 |
| [12] | 史殿习, 刘聪, 佘馥江, 张拥军. GPS拒止环境下基于定位置信度的多无人机协同定位方法 Cooperation Localization Method Based on Location Confidence of Multi-UAV in GPS-deniedEnvironment 计算机科学, 2022, 49(4): 302-311. https://doi.org/10.11896/jsjkx.210200106 |
| [13] | 彭冬阳, 王睿, 胡谷雨, 祖家琛, 王田丰. 视频缓存策略中QoE和能量效率的公平联合优化 Fair Joint Optimization of QoE and Energy Efficiency in Caching Strategy for Videos 计算机科学, 2022, 49(4): 312-320. https://doi.org/10.11896/jsjkx.210800027 |
| [14] | 欧阳卓, 周思源, 吕勇, 谭国平, 张悦, 项亮亮. 基于深度强化学习的无信号灯交叉路口车辆控制 DRL-based Vehicle Control Strategy for Signal-free Intersections 计算机科学, 2022, 49(3): 46-51. https://doi.org/10.11896/jsjkx.210700010 |
| [15] | 赵耿, 宋鑫宇, 马英杰. 混沌子载波调制的无人机安全数据链路 Secure Data Link of Unmanned Aerial Vehicle Based on Chaotic Sub-carrier Modulation 计算机科学, 2022, 49(3): 322-328. https://doi.org/10.11896/jsjkx.210200022 |
|
||
首页