Abstract: At present,most models of collision avoidance algorithms take ships as single agent to make collision avoidance decisions,without considering the coordinated avoidance between ships.In the scenario of multi-ship meeting,it will lead to poor avoidance effect by relying on single ships.Therefore,this paper proposes a softmax deep double deterministic policy gradients(SD3) multi-ship cooperative collision avoidance model based on improved twin delayed deep deterministic policy gradient(TD3).The time collision model and space collision model are constructed to quantitatively analyze the ship collision risk based on the time and space factors of ship navigation safety.On this basis,the ship domain model based on the situation of collision and the dynamic change of ship speed vector is used to qualitatively analyze the ship collision risk.The reward function is designed using the constraints of ship objective guidance,course angle change,course keeping,collision risk and international regulations for preventing collisions at sea(COLREGs),combined with the typical encounter situation in COLREGS,the collision avoidance simulation is carried out for the encounter scene with multi-situation coexistence of encounter,head-on,chase and cross encounter.Ablation experiment shows that the softmax operator improves the performance of SD3 algorithm,making it have better decision-ma-king effect in ship coordinated collision avoidance and compared with other reinforcement learning algorithms for learning efficiency and learning effect.Experimental results show that the SD3 algorithm can effectively make accurate collision avoidance decisions and outperform other reinforcement learning algorithms in performance in complex multi-situation encounter scenarios.

Key words: Vessel encounter, Coordinated collision avoidance, Intelligent decision-making, Twin delayed deep deterministic policy gradient(TD3), Softmax deep double deterministic policy gradients(SD3), Reinforcement learning