关键词: 深度伪造, 视频取证, 双流网络, 注意力机制, RGB分支, 频率域分支

Abstract: Deepfake is a kind of deep network model based on generative adversarial networks(GAN).It uses the source and target faces to generate highly realistic face videos that are difficult to identify.If some malicious person uses this technology to make fake videos and spread rumors on the Internet,it will infringe personal portrait right,cause adverse social impact,or even cause serious judicial disputes.In view of the serious threat brought by deepfake technology,many researchers at home and abroad pay close attention to the study of the deepfake detection technology,and have put forward some effective detection me-thods.The existing detection methods achieve good detection results in high-quality videos,but most videos in daily applications are usually compressed into low-quality versions through social software.However,most of the existing deepfake detection methods have a significant decline in detecting this kind of low-quality videos.Besides,the detection performances of existing methods are still unsatisfying in the case of cross datasets,limiting their real applications.To address this issue,this paper proposes a dual-flow network structure based on Xception model under multiple attention mechanism.The network structure includes an RGB branch using multiple attention mechanism and a frequency-domain branch for capturing low quality video artifacts.Based on our research,it is found that the tiny difference between real images and fake images tends to concentrates in some local area.The RGB branch under the multiple attention mechanism makes the model focus on different regions of the face,so it can get the glo-bal features aggregated by the low-level texture and high-level semantic features under the guidance of the attention map.Combined with the RGB branch,the discrete cosine transform(DCT) is introduced in the frequency domain branch to provide complementary feature representation,which can reflect subtle forgery traces or compression errors.Specifically,the proposed algorithm firstly extracts a large number of face frames from videos by face extractor algorithm,and feeds these face frames into the two-branch network model.The frequency branch decomposes the spectrum of images with three combined filters that provide additional learnable parts.In the RGB branch,the first three layers of the backbone network extract shallow features including texture information etc.Then the attention module makes the model attend to the shallow information from different local areas.The shallow information is then fed to the attention pooling layer to aggregate with the high-level semantic features from the rest layers of the backbone network.Finally,the network merges the feature vectors from both the RGB-branch and the frequency branch to obtain the final discriminant result.The combination of these two branches can significantly improve the detection performance of the model in cross database scenes and low-quality video sets.In order to verify the effectiveness of the proposed network structure,a large number of comparative experiments are conducted on three public datasets,including FaceForensics++,Celeb DF and DFDC.In the low-quality part of FaceForensics++ dataset,the AUC(Area Under the Curve) can reach 0.9271.In the video level,the detection accuracy of low-quality and high-quality videos can reach 93.84% and 99.69%,respectively.Experimental results show that the proposed algorithm outperforms the existing detection algorithms in low-quality video sets as well as in cross dataset scenes.It verifies that the combination of dual-stream features including the RGB branch and the frequency branch can improve the robustness of the detection method,especially in low-quality video sets and in cross dataset scenes.

Key words: Deepfake, Video forensics, Dual stream network, Attention mechanism, RGB branch, Frequency branch