Abstract: Value factorization is a popular method to solve cooperative multi-agent deep reinforcement learning problems,which factorizes joint value function into individual value functions according to IGM principle.In this method,agents select actions only according to individual value functions based on local observation,which leads to agents cannot effectively use global information to learn strategy.Although many value factorization algorithms extract the features of global state to weight individual value functions by many approaches,including attention mechanism,super network,and et al,so as to indirectly utilize global information to train agents,but this utilization is pretty limited.In a complex environment,it is difficult for agents to learn effective stra-tegies and their learning efficiency is poor.In order to improve agents' policy learning ability,an optimized value factorization method based on state estimation(SE-VF) is put forward,which introduces a state network to extract the features of global state and get a state value,and then take state loss value as part of the loss function to update agents network parameters,so as to optimize the strategy selection process of agents.Experimental results show that SE-VF performs better than QMIX and other baselines in multiple scenarios of the StarCraft 2 micromanagement mission test platform.

Key words: State estimation, Value factorization, Multi-agent reinforcement learning, Deep reinforcement learning