Abstract: Windows domain is a prime target for intranet penetration.However,the scenarios and methods of Windows domain penetration testing are very different from those of conventional intranet penetration..Existing research on intelligent path discovery is not suitable for the intricacies of Windows domain environments.Therefore,the current conventional intelligent path discovery research is not applicable to the Windows domain environment.In order to enhance the security protection of Windows domain,an automatic generation method of Windows domain penetration testing path based on deep reinforcement learning is proposed.Firstly,Windows domain penetration testing scenario is modeled as Markov decision process,and a simulator suitable for reinforcement learning is designed through Gymnasium of OpenAI.Secondly,in response to the challenge of limited exploration in large action and observation spaces,prior knowledge is leveraged to eliminate redundant actions and streamline the observation space.Lastly,the virtual machine technology is used to deploy the Windows domain environment in the small server,and the NDD-DQN is used as the basic algorithm to realize the whole process automation from information collection,model construction to path generation in the real environment.Experimental results show that the proposed method exhibit effective simulation and training effect in complex,real-world Windows domain environments.

Key words: Penetration testing, Windows domain, Deep reinforcement learning, DQN algorithm, Attack path