Abstract: Modular robots are composed of a certain number of standard modules with independent functions.At present,self reconfiguration is a hot and difficult problem in the field of modular robot research.For complex problems,the traditional graph theory algorithm or search algorithm cannot find its optimal solution in polynomial time,and the complexity increases exponentially with the increase of the number of modules.From the perspective of deep reinforcement learning of swarm agents,the research regards each isomorphic module as a single agent with learning and perception ability,and proposes a modular robot self reconfiguration algorithm based on QMIX.For this algorithm,a new type of reward function is designed and the parallel movement of the agent on the basis of limiting the action space of the agents is realized,which solves the problem of coordination and cooperation between multiple agents to a certain extent,thereby realizing the transition from the initial configuration to the target configuration.In addition,in experiments,9 modules are taken as examples to compare the success rate and average steps between this algorithm and the traditional search algorithm based on A^*.Experimental results show that when the time step limit is reasonable,the success rate of the modular robot self-reconfiguration algorithm based on QMIX can reach more than 95%,and the average number of steps of the two algorithms is about 12 steps.The QMIX self-reconfiguration algorithm can approach the effect of the traditional algorithm.

Key words: Modular robot, Self reconfiguration, Swarm agent collaboration, Deep reinforcement learning, Configuration space and action space