关键词: 航天器交会制导, 障碍物避撞, 安全强化学习, 近端策略优化, 深度确定性策略梯度

Abstract: With the increasing complexity of spacecraft rendezvous and docking tasks,the requirements for its efficiency,autonomy and reliability are highly demanded.In recent years,the introduction of reinforcement learning technology to solve the problem of spacecraft rendezvous and guidance has become an international frontier hotspot.Obstacle avoidance is critical for safe spacecraft rendezvous,and the general reinforcement learning algorithm does not impose safety restrictions on space exploration,which make the design of spacecraft rendezvous guidance policy challenging.This paper proposes a spacecraft rendezvous guidance method based on safe reinforcement learning.First,a Markov model of autonomous spacecraft rendezvous in collision avoidance scenarios is designed,a reward mechanism based on obstacle warning and collision avoidance restraint is proposed,and thus a safe reinforcement learning framework for solving spacecraft rendezvous guidance strategy is established.Second,with the framework of safe reinforcement learning,guidance policies are generated based on two deep reinforcement learning algorithms,proximal po-licy optimization(PPO) and deep deterministic policy gradient(DDPG).Experimental results show that the method can effectively avoid obstacle and complete the rendezvous with high accuracy.In addition,the performance and generalization ability of the two algorithms are analyzed,which proves the effectiveness of the proposed method.

Key words: Spacecraft rendezvous guidance, Obstacle avoidance, Safe reinforcement learning, Proximal policy optimization, Deep deterministic policy gradient