计算机科学

计算机科学 ›› 2026, Vol. 53 ›› Issue (2): 406-415.doi: 10.11896/jsjkx.250200092

• 计算机网络 • 上一篇    下一篇

低轨卫星网络中基于深度强化学习的航空器任务卸载策略

李芳, 袁宝淳, 沈航, 王天荆, 白光伟   

  1. 南京工业大学计算机与信息工程学院(人工智能学院) 南京 211816
  • 收稿日期:2025-02-24 修回日期:2025-05-19 发布日期:2026-02-10
  • 通讯作者: 沈航(hshen@njtech.edu.cn)
  • 作者简介:(202261220027@njtech.edu.cn)
  • 基金资助:
    国家自然科学基金 (61502230,61501224);江苏省自然科学基金 (BK20201357);江苏省“六大人才高峰”高层次人才项目(RJFW-020);江苏省研究生科研与实践创新计划(SJCX24_0566)

Deep Reinforcement Learning-based Aircraft Task Offloading in Low Earth Orbit Satellite Networks

LI Fang, YUAN Baochun, SHEN Hang, WANG Tianjing, BAI Guangwei   

  1. College of Computer and Information Engineering(College of Artificial Intelligence),Nanjing Tech University,Nanjing 211816,China
  • Received:2025-02-24 Revised:2025-05-19 Online:2026-02-10
  • About author:LI Fang,born in 1999,postgraduate.Her main research interest is intelligent network computing.
    SHEN Hang,born in 1984,Ph.D,asso-ciate professor,master’s supervisor,is a senior member of CCF(No.19088S).His main research interest is space-air-ground integrated networks.
  • Supported by:
    National Natural Science Foundation of China(61502230,61501224),Natural Science Foundation of Jiangsu Province,China(BK20201357),Six Talent Peaks Project in Jiangsu Province(RJFW-020) and Postgraduate Research & Practice Innovation Program of Jiangsu Province(SJCX24_0566).

PDF (PC)

65

摘要： 低地球轨道(LEO)卫星通信具有传输距离远、覆盖范围广、不受地形地貌限制等优点,已成为民航运输业和通用航空业的重要通信手段。然而,低轨卫星网络是一个高度异构和动态的环境,卫星节点的移动性、通信链路的复杂性、航空器时空分布不均和多种业务并存等特点使得任务卸载和资源分配面临许多挑战性问题。为此,提出了一种基于双深度强化学习(Double Deep Reinforcement Learning,DDRL)的航空器任务卸载方法,目的是最大化系统整体效用。首先,系统效用最大化问题被建模为一个任务卸载和资源分配的联合优化问题,同时考虑LEO卫星的计算能力和覆盖时间。接下来,将问题转换为马尔可夫决策过程,利用双重深度Q网络(Dual Deep Q Network,DDQN)算法学习最优的任务卸载决策,并在此基础上使用时间差分三重策略梯度(Time Difference Triple Policy Gradient,TD3)算法以获得最优资源分配策略。仿真实验表明,在不同的计算资源和通信资源下,所提出的方案在系统效用上优于其他基准方案,证明了所提框架的可用性。

关键词: 低轨卫星网络, 任务卸载, 资源分配, 深度强化学习

Abstract: LEO satellite communication has the advantages of long transmission distance,wide coverage,and is not restricted by terrain.It has become an important communication method for the civil aviation transportation and general aviation industries.However,the low-orbit satellite network is a highly heterogeneous and dynamic environment.The mobility of satellite nodes,the complexity of communication links,uneven spatial and temporal distribution of aircraft,and the coexistence of multiple services make task offloading and resource allocation face many challenges.To this end,this paper proposes an aircraft task offloading method based on DDRL,with the purpose of maximizing the overall effectiveness of the system.Firstly,the system utility maximization problem is modeled as a joint optimization problem of task offloading and resource allocation,taking into account the computing power and coverage time of LEO satellites.Next,the problem is transformed into a Markov decision process,using DDQN algorithm to learn the optimal task offloading decision,and based on this,TD3 is used to obtain the optimal resource allocation strategy.Simulation experiments show that under different computing resources and communication resources,the proposed scheme is better than other benchmark schemes in terms of system utility,proving the usability of the proposed framework.

Key words: LEO satellite network, Task offloading, Resource allocation, Deep reinforcement learning

中图分类号: 

  • TP393
[1]MAO Y,YOU C,ZHANG J,et al.A survey on mobile edge computing:The communication perspective[J].IEEE communications surveys & tutorials,2017,19(4):2322-2358.
[2]DENBY B,LUCIA B.Orbital edge computing:Machine infe-rence in space[J].IEEE Computer Architecture Letters,2019,18(1):59-62.
[3]WEI J,HAN J,CAO S.Satellite IoT edge intelligent computing:A research on architecture[J].Electronics,2019,8(11):1247.
[4]CAO S,ZHAO Y,WEI J,et al.Space-based cloud-fog computing architecture and its applications[C]//2019 IEEE World Congress on Services(SERVICES).IEEE,2019:166-171.
[5]GUO J,DU Y.Fog service in space information network:Architecture,use case,security and challenges[J].IEEE Access,2020,8:11104-11115.
[6]WEI J,CAO S.Application of edge intelligent computing in sa-tellite Internet of Things[C]//2019 IEEE International Confe-rence on Smart Internet of Things(SmartIoT).IEEE,2019:85-91.
[7]WANG Y,YANG J,GUO X,et al.Satellite edge computing for the internet of things in aerospace[J].Sensors,2019,19(20):4375.
[8]XIE R,TANG Q,WANG Q,et al.Satellite-terrestrial integrated edge computing networks:Architecture,challenges,and open issues[J].IEEE Network,2020,34(3):224-231.
[9]LI C,ZHANG Y,HAO X,et al.Jointly optimized request dispatching and service placement for MEC in LEO network[J].China Communications,2020,17(8):199-208.
[10]CAO B,ZHANG J,LIU X,et al.Edge-cloud resource scheduling in space-air-ground-integrated networks for internet of vehicles[J].IEEE Internet of Things Journal,2021,9(8):5765-5772.
[11]CUI G,LONG Y,XU L,et al.Joint offloading and resource allocation for satellite assisted vehicle-to-vehicle communication[J].IEEE Systems Journal,2020,15(3):3958-3969.
[12]WANG B,FENG T,HUANG D.A joint computation offloading and resource allocation strategy for LEO satellite edge computing system[C]//2020 IEEE 20th International Conference on Communication Technology(ICCT).IEEE,2020:649-655.
[13]CUI G,LI X,XU L,et al.Latency and energy optimization for MEC enhanced SAT-IoT networks[J].IEEE Access,2020,8:55915-55926.
[14]TRAN T X,POMPILI D.Joint task offloading and resource allocation for multi-server mobile-edge computing networks[J].IEEE Transactions on Vehicular Technology,2018,68(1):856-868.
[15]LI N,YUE C F,GUO H B,et al.Design of large scale satellite cluster domain control strategy[J].Chinese Space Science and Technology,2023,43(1):18-28.
[16]CASONI M,GRAZIA C A,KLAPEZ M,et al.Integration of satellite and LTE for disaster recovery[J].IEEE Communications Magazine,2015,53(3):47-53.
[17]WANG F,JIANG D,QI S,et al.Fine-grained resource management for edge computing satellite networks[C]//2019 IEEE Global Communications Conference(GLOBECOM).IEEE,2019:1-6.
[18]SU Y,LIU Y,ZHOU Y,et al.Broadband LEO satellite communications:Architectures and key technologies[J].IEEE Wireless Communications,2019,26(2):55-61.
[19]TANG Q,FEI Z,LI B,et al.Computation offloading in LEOsatellite networks with hybrid cloud and edge computing[J].IEEE Internet of Things Journal,2021,8(11):9169176.
[20]CUI G,LONG Y,XU L,et al.Joint offloading and resource allocation for satellite assisted vehicle-to-vehicle communication[J].IEEE Systems Journal,2020,15(3):3958-3969.
[21]DE SANCTIS M,CIANCA E,ARANITI G,et al.Satellite communications supporting internet of remote things[J].IEEE Internet of Things Journal,2015,3(1):113-123.
[22]CAO X,YANG B,SHEN Y,et al.Edge-assisted multi-layer offloading optimization of LEO satellite-terrestrial integrated networks[J].IEEE Journal on Selected Areas in Communications,2022,41(2):381-398.
[23]SONI G,SHARMA M.Performance Evaluation of a Free Space Optical Link-Based Inter Satellite Link(ISL) across Low Earth Orbit(LEO)[C]//2022 2nd International Conference on Power Electronics & IoT Applications in Renewable Energy and Its Control(PARC).IEEE,2022:1-5.
[24]ZHANG H,LIU R,KAUSHIK A,et al.Satellite edge computing with collaborative computation offloading:An intelligent deep deterministic policy gradient approach[J].IEEE Internet of Things Journal,2023,10(10):9092-9107.
[25]GAO X,LIU R,KAUSHIK A.Virtual network function placement in satellite edge computing with a potential game approach[J].IEEE Transactions on Network and Service Management,2022,19(2):1243-1259.
[26]GAO X,LIU R,KAUSHIK A,et al.Dynamic resource allocation for virtual network function placement in satellite edge clouds[J].IEEE Transactions on Network Science and Engineering,2022,9(4):2252-2265.
[27]LI Q,WANG S,MA X,et al.Service coverage for satellite edge computing[J].IEEE Internet of Things Journal,2021,9(1):695-705.
[28]CAO X,YANG B,SHEN Y,et al.Edge-assisted multi-layer offloading optimization of LEO satellite-terrestrial integrated networks[J].IEEE Journal on Selected Areas in Communications,2022,41(2):381-398.
[29]WU Y C,DINH T Q,FU Y,et al.A hybrid DQN and optimization approach for strategy and resource allocation in MEC networks[J].IEEE Transactions on Wireless Communications,2021,20(7):4282-4295.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市两江新区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发