计算机科学 ›› 2025, Vol. 52 ›› Issue (8): 326-334.doi: 10.11896/jsjkx.240600104

• 人工智能 • 上一篇    下一篇

基于改进Dueling-DQN的多无人机路径规划算法

付文浩, 葛礼勇, 汪文, 张淳   

  1. 南京邮电大学计算机学院 南京 210023
  • 收稿日期:2024-06-17 修回日期:2024-09-22 出版日期:2025-08-15 发布日期:2025-08-08
  • 通讯作者: 张淳(zhc1088@njupt.edu.cn)
  • 作者简介:(zhc1088@njupt.edu.cn)
  • 基金资助:
    国家自然科学基金(61903198)

Multi-UAV Path Planning Algorithm Based on Improved Dueling-DQN

FU Wenhao, GE Liyong, WANG Wen, ZHANG Chun   

  1. College of Computer,Nanjing University of Posts and Telecommunications,Nanjing 210023,China
  • Received:2024-06-17 Revised:2024-09-22 Online:2025-08-15 Published:2025-08-08
  • About author:FU Wenhao,born in 2000,postgra-duate.His main research interests include UAV task assignment and material delivery.
    ZHANG Chun,born in 1985,Ph.D,associate professor.Her main research interests include swarm intelligence and Internet of Things.
  • Supported by:
    National Natural Science Foundation of China(61903198).

摘要: 为了解决多无人机在三维未知障碍环境中对动态目标追击的路径规划问题,将人工势场法与深度强化学习算法结合,提出一种基于改进dueling deep Q network(Dueling-DQN)的多无人机路径规划算法,用于解决多无人机合作捕捉动态目标的路径规划问题。首先,将人工势场法的思想融入到多无人机合作捕捉动态目标的训练奖励函数中,不仅解决了传统人工势场法复杂环境中表现不佳,易陷入局部最优的问题,同时解决了多无人机合作和无人机复杂环境避障问题。此外,为了使无人机之间能更好合作捕捉动态目标,设计了一种多无人机与动态目标的捕捉逃逸策略。仿真结果表明,与Dueling-DQN算法相比,提出的APF-Dueling-DQN算法有效降低了无人机航迹规划任务过程中发生碰撞的概率,缩短了捕捉动态目标所需规划路径长度。

关键词: 多无人机, Dueling-DQN, 路径规划, 避障, 人工势场, 捕捉逃逸

Abstract: To address the problem of path planning for multiple unmanned aerial vehicles(UAVs) in three-dimensional unknown obstacle environments when pursuing dynamic targets,this paper proposes a path planning algorithm based on an improved due-ling deep Q network(Dueling-DQN) combined with the artificial potential field method and deep reinforcement learning algorithm.This is aimed at solving the problem of path planning for multiple UAVs cooperating to capture dynamic targets.Firstly,it incorporates the idea of the artificial potential field method into the training reward function for multiple UAVs cooperating to capture dynamic targets,which not only addresses the shortcomings of traditional artificial potential field methods in complex environments where they are prone to local optima,but also solves the problems of multi-UAV cooperation and UAV obstacle avoidance in complex environments.Additionally,to facilitate better cooperation among UAVs in capturing dynamic targets,a strategy for the capture and escape of dynamic targets by multiple UAVs is designed.Simulation results demonstrate that compared to Dueling-DQN algorithm,the proposed APF-Dueling-DQN algorithm effectively reduces the probability of collisions du-ring UAV trajectory planning tasks and shortens the planned path length required to capture dynamic targets.

Key words: UAVs, Dueling-DQN, Path planning, Obstacle avoidance, Artificial potential field, Capture and escape

中图分类号: 

  • TP181
[1]LI J Q,SUN T,HUANG X P,et al.A Memetic Path Planning Algorithm for Unmanned Air/Ground Vehicle Cooperative Detection Systems[J].IEEE Transactions on Automation Science and Engineering,2022,19(4):2724-2737.
[2]LI D C,YIN W P,WONG W E,et al.Quality-Oriented Hybrid Path Planning Based on A* and Q-Learning for Unmanned Aerial Vehicle[J].IEEE Access,2022,10:7664-7674.
[3]CHEN J,YE F,LI Y B.Travelling salesman problem for UAV path planning with two parallel optimization algorithms[C]//Progress in Electromagnetics Research Symposium.Singapore:IEEE,2017:832-837.
[4]XIONG C Y,XIN B,GUO M,et al.Multi-UAV 3D Path Planning in Simultaneous Attack[C]//IEEE International Confe-rence on Control and Automation ICCA.Singapore:IEEE,2020:500-505.
[5]THEILE M,BAYERLEIN H,NAI R,et al.UAV Path Planning using Global and Local Map Information with Deep Reinforcement Learning[C]//2021 20th International Conference on Advanced Robotics(ICAR).New York:IEEE,2021:539-546.
[6]GU Z L,LIU Y,SUN W B,et al.UAV Dynamic Route Planning Algorithm Based on RRT[J].Computer Science,2023,50(S1):65-69.
[7]LI Y Y,CHEN W Y,FU B,et al.Research on Dynamic Target Search for Multi-UAV Based on Cooperative Coevolution Motion-Encoded Particle Swarm Optimization[J].Applied Sciences-basel,2024,14(4):1326.
[8]ZHANG X Y,YUE W.Elite Dung Beetle Optimization Algorithm for MultiUAV Cooperative Search in Mountainous Environments[J].Journal of Bionic Engineering,2024,21(4):1677-1694.
[9]TANG J N,PENG Z X,LI M S,et al.Research on UAV path planning based on improved A-star algorithm [J].Electronic Measurement,2019,46(8):99-104.
[10]XIANG L H,WANG F Y,XU W J,et al.Dynamic UAV Swarm Collaboration for Multi-Targets Tracking Under Malicious Jamming:Joint Power,Path and Target Association Optimization[J].IEEE Transactions on Vehicular Technology,2024,73(4):5410-5425.
[11]YILDIZ B,ASLAN M F,DURDU A,et al.Consensus-based virtual leader tracking swarm algorithm with GDRRT*-PSO for path-planning of multiple-UAVs[J].Swarm and Evolutionary Computation,2024,88:101612.
[12]ZHANG W,SONG K,RONG X.Coarse-to- fine UAV targettracking with deep reinforcement learning[J].IEEE Transactions on Automation Science and Engineering,2019,16(4):1522-1530.
[13]ZHOU B,GUO Y,LI N,et al.UAV path planning based onguided reinforcement Q learning [J].Acta Aeronautica et Astronautica Sinica,2021,42(9):506-513.
[14]KOWNACKI C.Artificial Potential Field Based Trajectory Trackingfor Quadcopter UAV Moving Targets[J].Sensors,2024,24(4):1343.
[15]LI Y B,ZHANG Z T,SUN Q,et al.An improved ant colony algorithm for multiple unmanned aerial vehicles route planning[J].Journal of the Franklin Institute,2024,361(15):107060.
[16]WANG X,GURSOY M C,ERPEK T,et al.Learning-basedUAV path planning for data collection with integrated collision avoidance[J].IEEE Internet of Things Journal,2022,9(17):16663-16676.
[17]ZHANG J,LU Y,CHE L,et al.Moving-distance-minimized PSO for mobile robot swarm[J].IEEE Transactions on Cybernetics,2021,52(9):9871-9881.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!