计算机科学

计算机科学 ›› 2026, Vol. 53 ›› Issue (9): 375-384.doi: 10.11896/jsjkx.250700002

• 计算机网络 • 上一篇    下一篇

DTD3-AQM:基于信息时效性的卫星网络队列管理算法

魏德宾1, 王欣睿1, 杨力2, 潘成胜3   

  1. 1 大连大学信息工程学院 辽宁 大连 116622
    2 南京理工大学自动化学院 南京 210094
    3 南京邮电大学通信与网络技术国家工程研究中心 南京 210003
  • 收稿日期:2025-07-01 修回日期:2025-10-09 出版日期:2026-09-15 发布日期:2026-09-10
  • 通讯作者: 潘成胜(pancs@sina.com)
  • 作者简介:(weidebin@163.com)
  • 基金资助:
    国家自然科学基金(U21B2003,61931004)

DTD3-AQM:Satellite Network Queue Management Algorithm Based on Information Timeliness

WEI Debin1, WANG Xinrui1, YANG Li2, PAN Chengsheng3   

  1. 1 School of Information Engineering,Dalian University,Dalian,Liaoning 116622,China
    2 School of Automation,Nanjing University of Science and Technology,Nanjing 210094,China
    3 National Engineering Research Center for Communication and Network Technology,Nanjing University of Posts and Telecommunications,Nanjing 210003,China
  • Received:2025-07-01 Revised:2025-10-09 Published:2026-09-15 Online:2026-09-10
  • About author:WEI Debin,born in 1978,Ph.D,asso-ciate professor,is a member of CCF(No.22419M).His main research interests include space-ground integrated network transmission technology,traffic engineering and network optimization.
    PAN Chengsheng,born in 1962,Ph.D,professor,is a member of CCF(No.09154F).His main research interests include intelligent network traffic theory and key technologies.
  • Supported by:
    National Natural Science Foundation of China (U21B2003,61931004).

PDF (PC)

236

摘要： 低轨卫星通信网络具有覆盖广、传播延迟低等优势,但其动态变化的拓扑结构、带宽受限等特性可能对高时效性业务的服务质量造成显著影响。针对此问题,提出了一种融合双延迟深度确定性策略梯度(Twin Delayed Deep Deterministic Policy Gradient,TD3)与对决网络(Dueling Network)的智能主动队列管理算法DTD3-AQM。该算法引入卫星网络的时变拓扑矩阵与多维队列性能指标,设计多目标奖励机制,在实现拥塞控制的同时兼顾数据新鲜度。基于MATLAB-NS3联合平台在Linux环境下开展的仿真实验表明,DTD3-AQM在各项性能指标中实现了较优的综合权衡。在200个TCP源的条件下,与侧重系统性能的QueuePilot相比,DTD3-AQM将排队时延减少11.3 ms、信息年龄降低87.2 ms,虽吞吐量略有下降,但提升了拥塞控制能力与信息时效保障水平;相较于侧重信息新鲜度的DeepAAQM,DTD3-AQM在吞吐量方面提升3.1 Mbps、丢包率下降0.29%,但DeepAAQM在降低信息年龄方面仍具有显著优势。

关键词: 卫星网络, 主动队列管理, 信息年龄, 强化学习, 对决网络

Abstract: Low Earth Orbit(LEO) satellite communication networks offer advantages such as broad coverage and low propagation delay.However,their dynamically changing topologies and bandwidth constraints may significantly impact the quality of service for time- sensitive applications.To address this issue,this paper proposes an intelligent active queue management algorithm,DTD3-AQM,which integrates the twin delayed deep deterministic policy gradient(TD3) framework with the dueling network architecture.This algorithm introduces time-varying topology matrices and multi-dimensional queue performance metrics to design a multi-objective reward mechanism,achieving congestion control while simultaneously considering data freshness.Simulation experiments conducted on a MATLAB-NS3 joint platform under a Linux environment demonstrate that DTD3-AQM achieves an optimal balance across multiple performance metrics.Under conditions with 200 TCP sources,compared to the system-perfor-mance-oriented QueuePilot,DTD3-AQM reduces queuing delay by 11.3 ms and average age of information(AoI) by 87.2 ms.Although it incurs a slight drop in throughput,it enhances congestion control capability and improves information timeliness.Compared to the freshness-focused DeepAAQM,DTD3-AQM increases throughput by 3.1 Mbps and reduces packet loss rate by 0.29%,though DeepAAQM still holds a significant advantage in reducing AoI.

Key words: Satellite network, Active queue management, Age of information, Reinforcement learning, Dueling network

中图分类号: 

  • TP393
[1] SONG T,KYUNG Y.Deep-reinforcement-learning-based age-of-information-aware low-power active queue management for IoT sensor networks[J].IEEE Internet of Things Journal,2024,11(9):16700-16709.
[2] PAPPAS N,GUNNARSSON J,KRATZ L,et al.Age of information of multiple sources with queue management[C] //2015 IEEE International Conference on Communications(ICC).IEEE,2015:5935-5940.
[3] FANG Z,WANG J,JIANG C,et al.Average peak age of information in underwater information collection with sleep-scheduling[J].IEEE Transactions on Vehicular Technology,2022,71(9):10132-10136.
[4] XU C L,CHEN Z Y,TAO M X,et al.Wireless Multi-User Interactive Virtual Reality in Metaverse with Edge-Device Colla-borative Computing[J].IEEE Transactions on Wireless Communications,2025,27(7):6135-6150.
[5] SU Y,HUANG L,FENG C.QRED:A Q-learning-based active queue management scheme[J].Journal of Internet Technology,2018,19(4):1169-1178.
[6] BYUN D J,BARAS J S.Adaptive virtual queue random earlydetection in satellite networks[M] // Wireless Technology:Applications,Management,and Security.2009:63-82.
[7] YOUSIF A,HASSAN H,MUTTASHER G.Applying rein-forcement learning for random early detection algorithm in adaptive queue management systems[J].Indonesian Journal of Electrical Engineering and Computer Science,2022,26(3):1684.
[8] LIU J Y,WEI D B.Active queue management based on Q-lear-ning traffic predictor[C] //2022 International Conference on Cyber-Physical Social Intelligence(ICCSI).IEEE,2022:399-404.
[9] KIM M,JASEEMUDDIN M,ANPALAGAN A.Deep reinforce-ment learning based active queue management for IoT networks[J].Journal of Network and Systems Management,2021,29(3):34.
[10] FORERO P A,ZHANG P,RADOSEVIC D.Active queue-management policies for undersea networking via deep reinforcement learning[C] //OCEANS 2021:San Diego-Porto.IEEE,2021:1-8.
[11] DERY M,KRUPNIK O,KESLASSY I.Queuepilot:Revivingsmall buffers with a learned aqm policy[C] //IEEE INFOCOM 2023－IEEE Conference on Computer Communications.IEEE,2023:1-10.
[12] HUI T F,ZHAI S H,GONG F K,et al.Research status and development trends of satellite communication multiple access technology[J].Space Electronic Technology,2026,23(1):116-134.
[13] LIU Y,SHI Y,LI J G,et al.Reconnaissance and jamming methods for low earth orbit constellations[J].Space Electronic Technology,2026,23(2):1-8.
[14] YAN Y,WANG L H,TIAN Z,et al.Research on probabilistic routing broadcast protocol for low-orbit satellite networks[J].Space Electronic Technology,2026,23(1):135-144.
[15] GRAZIA C A,PATRICIELLO N,KLAPEZ M,et al.Mitigating congestion and bufferbloat on satellite networks through a rate-based AQM[C] //2017 IEEE International Conference on Communications(ICC).IEEE,2017:1-6.
[16] MA Y,ZHANG T,ZHANG J.An AQM algorithm for LEO sa-tellite network[C] //2011 4th IEEE International Symposium on Microwave,Antenna,Propagation and EMC Technologies for Wireless Communications.IEEE,2011:615-618.
[17] YATES R D,SUN Y,BROWN D R,et al.Age of information:An introduction and survey[J].IEEE Journal on Selected Areas in Communications,2021,39(5):1183-1210.
[18] SUTTON R S,BARTO A G.Reinforcement learning[J].Journal of Cognitive Neuroscience,1999,11(1):126-134.
[19] LILLICRAP T P,HUNT J J,PRITZEL A,et al.Continuouscontrol with deep reinforcement learning[J].arXiv:1509.02971,2015.
[20] FUJIMOTO S,HOOF H,MEGER D.Addressing function approximation error in actor-critic methods[C] //International Conference on Machine Learning.PMLR,2018:1587-1596.
[21] WANG Z,SCHAUL T,HESSEL M,et al.Dueling network architectures for deep reinforcement learning[C] //International Conference on Machine Learning.PMLR,2016:1995-2003.
[22] SHUKUR HH A,AHMAD Y A,YUNUS M S F M,et al.Latency Performance Evaluation of LEO Starlink and SES-12 GEO HTS Network Under Tropical Rainfall Conditions [J].IIUM Engineering Journal,2025,26(2):204-219.
[23] LAI Z,LIU C,WANG X,et al.SpaceRTC:Unleashing the low-latency potential of mega-constellations for real-time communications [C] //IEEE INFOCOM 2022－IEEE Conference on Computer Communications.IEEE,2022:1339-1348.
[24] ZHANG T.Toward automated vehicle teleoperation:Vision,opportunities,and challenges [J].IEEE Internet of Things Journal,2020,7(12):11347-11354.
[25] MA H,XU D,DAI Y,et al.An intelligent scheme for congestion control:When active queue management meets deep reinforcement learning[J].Computer Networks,2021,200:108515.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市两江新区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发