Abstract: With the development and widespread application of network technology,encrypted traffic has become a key technology for protecting user privacy.However,malware and attackers also use encrypted traffic to hide their behaviors and evade traditional network intrusion detection systems.Existing malicious encrypted traffic detection methods have some pro-blems.Statistics-based methods rely on expert experience for feature extraction,and features of different protocols cannot be generalized.Deep learning methods based on raw inputs have incomplete information and field padding data issues,leading to insufficient semantic representation of encrypted traffic interactions.To solve the above problems,this paper proposes a method called “conversation statistic encoder model(CSEM)”.The method draws on the transformer encoder model and introduces a new traffic packet feature parsing method,and it is different from the traditional mode of inputting byte streams into deep neural networks.The proposed method can construct fixed-length vector representations for each traffic packet without padding zeros,while avoiding dependence on specific encrypted protocols in the feature extraction process.A hybrid deep neural network is constructed to provide a new idea for malicious encrypted traffic detection.The proposed method is verified on the DataCon dataset and self- built dataset,and the experimental results on Datacon dataset show a recall of 0.991 1,precision of 0.940 7,and F1 score of 0.965 2,reaching the current best level,and the F1 score is 9% higher than that of the random forest model.

Key words: Conversation, Encrypted traffic detection, Encoder