关键词: 卫星网络, 软件定义网络, 多路径路由, 深度强化学习, 负载均衡

Abstract: In the satellite network,due to the influence of geographical location and people's living habits,the difference in the needs of users in the satellite coverage area will cause the load imbalance of the satellite network.A multi-path routing algorithm based onconvolutional double-delay deep deterministic policy gradient(CTD3-MR) is proposed for the above problem.Under the SDN structure,CTD3 is deployed in the controller as the agent,and the dynamically changing links' residual bandwidth,transmission delay,packet loss rate and spatiotemporal level are trained as the network state input agent,and the output action is used as the network link weight,and the weighted sum of the maximum link bandwidth utilization,average end-to-end delay and network packet loss rate is used as the reward function to adjust the action.After the agent training converges,the controller obtains the k-shortest path according to the network link weight output by the agent,and takes the path weight ratio as the path traffic allocation ratio to generate an optimal routing strategy and forward it to the satellite for multipath transmission.Finally,CTD3-MR is compared with TD3,TMR and ECMP routing algorithms.Experimental results show that compared with other routing algorithms,CTD3-MR reduces the average end-to-end delay by at least 7.64%,the packet loss rate by 28.65%,the maximum link bandwidth utilization by 11.44%,and the traffic distribution index by 5.82%,which improves the network load balancing performance.

Key words: Satellite networks, Software defined networking, Multipath routing, Deep reinforcement learning, Load balancing