Computer Science ›› 2025, Vol. 52 ›› Issue (8): 326-334.doi: 10.11896/jsjkx.240600104

• Artificial Intelligence • Previous Articles     Next Articles

Multi-UAV Path Planning Algorithm Based on Improved Dueling-DQN

FU Wenhao, GE Liyong, WANG Wen, ZHANG Chun   

  1. College of Computer,Nanjing University of Posts and Telecommunications,Nanjing 210023,China
  • Received:2024-06-17 Revised:2024-09-22 Online:2025-08-15 Published:2025-08-08
  • About author:FU Wenhao,born in 2000,postgra-duate.His main research interests include UAV task assignment and material delivery.
    ZHANG Chun,born in 1985,Ph.D,associate professor.Her main research interests include swarm intelligence and Internet of Things.
  • Supported by:
    National Natural Science Foundation of China(61903198).

Abstract: To address the problem of path planning for multiple unmanned aerial vehicles(UAVs) in three-dimensional unknown obstacle environments when pursuing dynamic targets,this paper proposes a path planning algorithm based on an improved due-ling deep Q network(Dueling-DQN) combined with the artificial potential field method and deep reinforcement learning algorithm.This is aimed at solving the problem of path planning for multiple UAVs cooperating to capture dynamic targets.Firstly,it incorporates the idea of the artificial potential field method into the training reward function for multiple UAVs cooperating to capture dynamic targets,which not only addresses the shortcomings of traditional artificial potential field methods in complex environments where they are prone to local optima,but also solves the problems of multi-UAV cooperation and UAV obstacle avoidance in complex environments.Additionally,to facilitate better cooperation among UAVs in capturing dynamic targets,a strategy for the capture and escape of dynamic targets by multiple UAVs is designed.Simulation results demonstrate that compared to Dueling-DQN algorithm,the proposed APF-Dueling-DQN algorithm effectively reduces the probability of collisions du-ring UAV trajectory planning tasks and shortens the planned path length required to capture dynamic targets.

Key words: UAVs, Dueling-DQN, Path planning, Obstacle avoidance, Artificial potential field, Capture and escape

CLC Number: 

  • TP181
[1]LI J Q,SUN T,HUANG X P,et al.A Memetic Path Planning Algorithm for Unmanned Air/Ground Vehicle Cooperative Detection Systems[J].IEEE Transactions on Automation Science and Engineering,2022,19(4):2724-2737.
[2]LI D C,YIN W P,WONG W E,et al.Quality-Oriented Hybrid Path Planning Based on A* and Q-Learning for Unmanned Aerial Vehicle[J].IEEE Access,2022,10:7664-7674.
[3]CHEN J,YE F,LI Y B.Travelling salesman problem for UAV path planning with two parallel optimization algorithms[C]//Progress in Electromagnetics Research Symposium.Singapore:IEEE,2017:832-837.
[4]XIONG C Y,XIN B,GUO M,et al.Multi-UAV 3D Path Planning in Simultaneous Attack[C]//IEEE International Confe-rence on Control and Automation ICCA.Singapore:IEEE,2020:500-505.
[5]THEILE M,BAYERLEIN H,NAI R,et al.UAV Path Planning using Global and Local Map Information with Deep Reinforcement Learning[C]//2021 20th International Conference on Advanced Robotics(ICAR).New York:IEEE,2021:539-546.
[6]GU Z L,LIU Y,SUN W B,et al.UAV Dynamic Route Planning Algorithm Based on RRT[J].Computer Science,2023,50(S1):65-69.
[7]LI Y Y,CHEN W Y,FU B,et al.Research on Dynamic Target Search for Multi-UAV Based on Cooperative Coevolution Motion-Encoded Particle Swarm Optimization[J].Applied Sciences-basel,2024,14(4):1326.
[8]ZHANG X Y,YUE W.Elite Dung Beetle Optimization Algorithm for MultiUAV Cooperative Search in Mountainous Environments[J].Journal of Bionic Engineering,2024,21(4):1677-1694.
[9]TANG J N,PENG Z X,LI M S,et al.Research on UAV path planning based on improved A-star algorithm [J].Electronic Measurement,2019,46(8):99-104.
[10]XIANG L H,WANG F Y,XU W J,et al.Dynamic UAV Swarm Collaboration for Multi-Targets Tracking Under Malicious Jamming:Joint Power,Path and Target Association Optimization[J].IEEE Transactions on Vehicular Technology,2024,73(4):5410-5425.
[11]YILDIZ B,ASLAN M F,DURDU A,et al.Consensus-based virtual leader tracking swarm algorithm with GDRRT*-PSO for path-planning of multiple-UAVs[J].Swarm and Evolutionary Computation,2024,88:101612.
[12]ZHANG W,SONG K,RONG X.Coarse-to- fine UAV targettracking with deep reinforcement learning[J].IEEE Transactions on Automation Science and Engineering,2019,16(4):1522-1530.
[13]ZHOU B,GUO Y,LI N,et al.UAV path planning based onguided reinforcement Q learning [J].Acta Aeronautica et Astronautica Sinica,2021,42(9):506-513.
[14]KOWNACKI C.Artificial Potential Field Based Trajectory Trackingfor Quadcopter UAV Moving Targets[J].Sensors,2024,24(4):1343.
[15]LI Y B,ZHANG Z T,SUN Q,et al.An improved ant colony algorithm for multiple unmanned aerial vehicles route planning[J].Journal of the Franklin Institute,2024,361(15):107060.
[16]WANG X,GURSOY M C,ERPEK T,et al.Learning-basedUAV path planning for data collection with integrated collision avoidance[J].IEEE Internet of Things Journal,2022,9(17):16663-16676.
[17]ZHANG J,LU Y,CHE L,et al.Moving-distance-minimized PSO for mobile robot swarm[J].IEEE Transactions on Cybernetics,2021,52(9):9871-9881.
[1] ZHANG Yongliang, LI Ziwen, XU Jiahao, JIANG Yuchen, CUI Ying. Congestion-aware and Cached Communication for Multi-agent Pathfinding [J]. Computer Science, 2025, 52(8): 317-325.
[2] LIU Qingyun, YOU Xiong, ZHANG Xin, ZUO Jiwei, LI Jia. Review of Path Planning Algorithms for Mobile Robots [J]. Computer Science, 2025, 52(6A): 240900074-10.
[3] YE Mingjun, WANG Shujian. UAV Path Planning Based on Improved Dung Beetle Optimization Algorithm [J]. Computer Science, 2025, 52(6A): 240900136-6.
[4] ZHAO Xuejian, YE Hao, LI Hao, SUN Zhixin. Multi-AGV Path Planning Algorithm Based on Improved DDPG [J]. Computer Science, 2025, 52(6): 306-315.
[5] KONG Chao, WANG Wei, HUANG Subin, ZHANG Yi, MENG Dan. Autonomous Obstacle Avoidance Method for Unmanned Surface Vehicles Based on ImprovedProximal Policy Optimization [J]. Computer Science, 2025, 52(4): 40-48.
[6] LIU Yi, QI Jie. IRRT*-APF Path Planning Algorithm Considering Kinematic Constraints of Unmanned Surface Vehicle [J]. Computer Science, 2024, 51(9): 290-298.
[7] WEI Shuxin, WANG Qunjing, LI Guoli, XU Jiazi, WEN Yan. Path Planning for Mobile Robots Based on Modified Adaptive Ant Colony Optimization Algorithm [J]. Computer Science, 2024, 51(6A): 230500145-9.
[8] MA Yinghong, LI Xu’nan, DONG Xu, JIAO Yi, CAI Wei, GUO Youguang. Fast Path Recovery Algorithm for Obstacle Avoidance Scenarios [J]. Computer Science, 2024, 51(6): 331-337.
[9] SUN Didi, LI Chaochao. Dynamic Path Planning Algorithm for Heterogeneous Groups in Aircraft Carrier Aviation SupportOperations [J]. Computer Science, 2024, 51(3): 226-234.
[10] AN Yang, WANG Xiuqing, ZHAO Minghua. Mobile Robots' Path Planning Method Based on Policy Fusion and Spiking Deep ReinforcementLearning [J]. Computer Science, 2024, 51(11A): 240100211-11.
[11] GU Wei, DUAN Jing, ZHANG Dong, HAO Xiaowei, XUE Honglin, AN Yi , DUAN Jie. Prediction of Spatial and Temporal Distribution of Electric Vehicle Charging Loads Based on Joint Data and Modeling Drive [J]. Computer Science, 2024, 51(11A): 231100110-6.
[12] WANG Ziyang, WANG Jia, XIONG Mingliang, WANG Wentao. Intelligent Penetration Path Based on Improved PPO Algorithm [J]. Computer Science, 2024, 51(11A): 231200165-6.
[13] LI Cheng’en, ZHU Dongjun, HE Jieyan, HAN Lansheng. Intelligent Penetration Path Planning and Solution Optimization Based on Reinforcement Learning [J]. Computer Science, 2024, 51(11): 329-339.
[14] LIU Xingguang, ZHOU Li, ZHANG Xiaoying, CHEN Haitao, ZHAO Haitao, WEI Jibo. Edge Intelligent Sensing Based UAV Space Trajectory Planning Method [J]. Computer Science, 2023, 50(9): 311-317.
[15] XING Linquan, XIAO Yingmin, YANG Zhibin, WEI Zhengmin, ZHOU Yong, GAO Saijun. Spacecraft Rendezvous Guidance Method Based on Safe Reinforcement Learning [J]. Computer Science, 2023, 50(8): 271-279.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!