Abstract: The unbalanced burden on the nodes nearing the ground station pose challenges on the multi-hop data transmission in aerial sensor networks(ASNs).In order to achieve reliable and efficient multi-hop data transmission in ASNs,a reinforcement-learning based queue-efficient geographic routing(RLQE-GR) protocol is proposed.The RLQE-GR protocol maps routing problem into the general reinforcement learning(RL) framework,where each UAV is treated as one state and each successful packet forwarding is treated as one action.Based on the framework,the RLQE-GR protocol designs a reward function related to geographical location,link quality and available transmission queue length.Then,the Q-function is employed to converge all the sta-teaction values(Q-values),and each packet is forwarded based on potential state-action values.To converge all Q values and minimize performance deterioration during the convergence process,a beacon mechanism is employed in RLQE-GR protocol.In contrast to existing geographic routing protocols,the RLQE-GR protocol simultaneously takes the queue utilization,link quality and relative distance into consideration for forwarding packets.This makes the RLQE-GR protocol achieve load balancing,meanwhile not introducing strict performance deteriorations on routing hop and link quality.Moreover,due to the near-optimization character of RL theory,the RLQE-GR protocol can achieve routing performance optimization on packet delivery ratio and end-to-end delay.

Key words: Aerial sensor network, Beacon mechanism, Geographic routing protocol, Reinforcement learning, Reward function