Abstract: Vulnerability mining is the main research direction in the field of computer software security,in which fuzz testing is an important dynamic mining method.In order to solve the problems such as time-consuming and low efficiency of fuzz testing caused by the large volume of assembly code,a novel binary code vulnerability mining technology based on deep reinforcement learning is proposed.The fuzz testing process is modeled as a multi-step Markov decision-making process oriented to reinforcement learning.The selection of fuzz testing mutation strategy is optimized by building a deep reinforcement learning model to achieve dynamic optimization.Then design and build a binary code fuzz testing platform based on deep reinforcement learning,use AFL to implement fuzz testing environment,and use Keras RL2 library and OpenAI Gym framework to implement deep reinforcement learning algorithm and reinforcement learning environment.Finally,the effectiveness and applicability of the proposed method and testing platform are verified through experimental analysis.Experimental results show that the deep reinforcement learning model can assist the fuzz testing process to quickly cover more paths,expose more vulnerabilities and defects,and significantly improve the efficiency of binary code vulnerability mining and location.

Key words: Binary code, Vulnerability mining, Fuzz testing, Deep reinforcement learning, Testing platform