Abstract: In the imitation learning method of behavior cloning(BC),an agent tends to take random actions when encountering states that are not covered by expert data.This deviation from the expert policy leads to what is known as compounding error,a critical factor affecting the performance of BC.To address this issue,this paper first establishes that BC can be regarded as a simplified form of twice learning.Furthermore,in discrete action environments,BC primarily focuses on aligning with the expert-selected actions while ignoring probability information associated with other actions,resulting in incomplete extraction of expert knowledge.Inspired by twice learning,this paper proposes an enhanced version of BC,termed complete behavior cloning(CBC),which aims to leverage a more comprehensive set of information from expert data.To validate the effectiveness of this approach,this paper designs multiple comparative experiments.The results demonstrate that CBC not only mitigates compounding error but also exhibits high transferability across different devices,enhanced robustness to noise,and reduced dependency on expert data.These findings suggest that BC can become highly practical and computationally efficient with only minor modifications.More-over,the experimental results further reinforce the guiding role and effectiveness of twice learning in reinforcement learning problems.

Key words: Imitation learning, Behavior cloning, Compounding error, Twice learning, Information extraction