Abstract: In RoboCup Keepaway task,players can gain good high-level strategy with reinforcement learning.However,as Keepaway tasks have very huge state space,normal reinforcement learning requires a great many searching steps to converge,and needs very long time.To solve this problem,for 5v4 scale Keepaway task,policy reuse technique is applied to the reinforcement learning procedure of takers’ high-level decision to achieve transfer learning.The transferring plan along with the map of state and action space between 4v3 and 5v4 task were rationally designed.Then a policy reuse based algorithm was stated.Experiments show that after the same training time for 5v4 scale task,takers get shorter task finish time and higher stealing success rate during transfer learning than in normal reinforcement learning.So there are better policies learned by transfer learning.Transfer learning needs much less training time than normal reinforcement learning to get the same policy level.

Key words: RoboCup soccer,Keepaway, Stealing police,Policy reuse,Transfer learning