基于非关键掩码和注意力机制的深度伪造人脸篡改视频检测方法

doi:10.11896/jsjkx.221100109

Abstract

Abstract: Since the introduction of Deepfake technology,its illegal application has caused a bad impact on individuals,society and national security,and there are huge hidden dangers.Therefore,deep fake detection for face video is a hot and difficult problem in the field of computer vision.In view of the above problems,this paper proposes a deepfake video detection method based on non-critical mask and CA_S3D Model.It firstly divides the face image into key areas and non-critical regions,and improves the attention of the deep neural network to the key areas of the face image through the mask processing of the non-critical areas,and reduces the influence and interference of irrelevant information on the deep neural network.Then it introduces the contextual attention module in the S3D network,which enhances the ability to capture the long-range dependence of sample data information and improves the attention to key channels and features.Experimental results show that the proposed method improves the perfor-mance of the deep neural network on the DFDC dataset,the accuracy rate increases from 83.85% to 90.10%,and the AUC value increases from 0.931 to 0.979.By comparing with the existing deepfake video detection methods,the performance of the proposed method is better than that of the existing methods,which verifies its effectiveness.

Key words: Deepfake, Deepfake detection, Image mask, 3D CNNs, Mechanism of attention

CLC Number:

TP391.41

YU Yang, YUAN Jiabin, CAI Jiyuan, ZHA Keke, CHEN Zhangyu, DAI Jiawei, FENG Yuxiang. Deepfake Face Tampering Video Detection Method Based on Non-critical Masks and AttentionMechanism[J].Computer Science, 2023, 50(11): 160-167.

References

[1]KINGMA D P,WELLING M.Auto-Encoding Variational Bayes[EB/OL].https://arxiv.org/abs/1312.6114.
[2]GOODFELLOW I J,POUGET-ABADIE J,MIRZA M,et al.Generative Adversarial Nets[C]//Proceedings of the 27th International Conference on Neural Information Processing Systems－Volume 2.Cambridge,MA,USA:MIT Press,2014:2672-2680.
[3]AKHTAR Z,DASGUPTA D.A Comparative Evaluation of Local Feature Descriptors for DeepFakes Detection[C]//2019 IEEE International Symposium on Technologies for Homeland Security(HST 2019).2019:1-5.
[4]ZHANG Y,ZHENG L,THING V L L.Automated face swapping and its detection[C]//2017 IEEE 2nd International Confe-rence on Signal and Image Processing(ICSIP).2017:15-19.
[5]KORSHUNOV P,MARCEL S.DeepFakes:a New Threat toFace Recognition?Assessment and Detection[EB/OL].[2021-08-23].http://arxiv.org/abs/1812.08685.
[6]Faceswap[CP/OL].https://github.com/deepfakes/faceswap.
[7]DFaker[CP/OL].https://github.com/dfaker/df.
[8]DeepFaceLab[CP/OL].https://github.com/iperov/DeepFace-Lab.
[9]Faceswap-GAN[CP/OL].https://github.com/shaoanlu/faces-wap-GAN.
[10]NIRKIN Y,KELLER Y,HASSNER T.FSGAN:Subject Agnostic Face Swapping and Reenactment[C]//2019 IEEE/CVF International Conference on Computer Vision(ICCV).2019:7183-7192.
[11]KORSHUNOVA I,SHI W,DAMBRE J,et al.Fast Face-SwapUsing Convolutional Neural Networks[C]//2017 IEEE International Conference on Computer Vision(ICCV).2017:3697-3705.
[12]MATERN F,RIESS C,STAMMINGER M.Exploiting Visual Artifacts to Expose Deepfakes and Face Manipulations[C]//2019 IEEE Winter Applications of Computer Vision Workshops(WACVW).IEEE,2019.
[13]LI Y,YANG X,SUN P,et al.Celeb-DF:A Large-Scale Challenging Dataset for DeepFake Forensics[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).2020:3204-3213.
[14]YU N,DAVIS L,FRITZ M.Attributing Fake Images to GANs:Learning and Analyzing GAN Fingerprints[C]//2019 IEEE/CVF International Conference on Computer Vision(ICCV).2019:7555-7565.
[15]AGARWAL S,FARID H,GU Y,et al.Protecting World Lea-ders Against Deep Fakes[C]//CVPR Workshops.2019:38-45.
[16]LI Y,CHANG M C,LYU S.In Ictu Oculi:Exposing AI Created Fake Videos by Detecting Eye Blinking[C]//2018 IEEE International Workshop on Information Forensics and Security(WIFS).2018:1-7.
[17]GUERA D,DELP E J.Deepfake Video Detection Using Recurrent Neural Networks[C]//2018 15th IEEE International Conference on Advanced Video and Signal Based Surveillance(AVSS).IEEE,2018.
[18]LI Y,LYU S.Exposing DeepFake Videos By Detecting FaceWarping Artifacts[C]//IEEE Conference on Computer Vision and Pattern Recognition Workshops(CVPRW).2019.
[19]CHUGH K,GUPTA P,DHALL A,et al.Not Made for Each Other- Audio-Visual Dissonance-Based Deepfake Detection and Localization[C]//Proceedings of the 28th ACM International Conference on Multimedia.New York,NY,USA:Association for Computing Machinery,2020:439-447.
[20]SCHROFF F,KALENICHENKO D,PHILBIN J.FaceNet:Aunified embedding for face recognition and clustering[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition(CVPR).IEEE,2015.
[21]O’TOOLE A J,JONATHON PHILLIPS P,JIANG F,et al.Face Recognition Algorithms Surpass Humans Matching Faces Over Changes in Illumination[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2007,29(9):1642-1646.
[22]NGUYEN H H,FANG F,YAMAGISHI J,et al.Multi-task Learning for Detecting and Segmenting Manipulated Facial Images and Videos[C]//2019 IEEE 10th International Conference on Biometrics Theory,Applications and Systems(BTAS).2019:1-8.
[23]AMERINI I,GALTERI L,CALDELLI R,et al.Deepfake Video Detection through Optical Flow Based CNN[C]//2019 IEEE/CVF International Conference on Computer Vision Workshop(ICCVW).IEEE,2019.
[24]SUN D,YANG X,LIU M Y,et al.PWC-Net:CNNs for Optical Flow Using Pyramid,Warping,and Cost Volume[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.2018:8934-8943.
[25]ALPARONE L,BARNI M,BARTOLINI F,et al.Regularization of optic flow estimates by means of weighted vector median filtering[J].IEEE Transactions on Image Processing,1999,8(10):1462-1467.
[26]LUO Z,KAMATA S I,SUN Z.Transformer and Node-Com-pressed Dnn Based Dual-Path System For Manipulated Face Detection[C]//2021 IEEE International Conference on Image Processing(ICIP).2021:3882-3886.
[27]XIE S,SUN C,HUANG J,et al.Rethinking SpatiotemporalFeature Learning:Speed-Accuracy Trade-Offs in Video Classification[C]//Computer Vision(ECCV 2018):15th European Conference.Munich,Germany,2018:318-335.
[28]CARREIRA J,ZISSERMAN A.Quo Vadis,Action Recogni-tion? A New Model and the Kinetics Dataset[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition(CVPR).2017:4724-4733.
[29]SZEGEDY C,LIU W,JIA Y,et al.Going deeper with convolutions[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition(CVPR).2015:1-9.
[30]CAO Y,XU J,LIN S,et al.GCNet:Non-local Networks Meet Squeeze-Excitation Networks and Beyond[EB/OL].[2022-09-28].http://arxiv.org/abs/1904.11492.
[31]NEHATE C,DALIA P,NAIK S,et al.Exposing DeepFakesusing Siamese Training[C]//2022 IEEE India Council International Subsections Conference(INDISCON).2022:1-6.
[32]COCCOMINI D A,MESSINA N,GENNARO C,et al.Combining EfficientNet and Vision Transformers for Video Deepfake Detection[C]//Image Analysis and Processing(ICIAP 2022).2022:219-229.
[33]WANG Z,LI X,NI R,et al.Attention Guided Spatio-Temporal Artifacts Extraction for Deepfake Detection[C]//Pattern Re-cognition and Computer Vision.2021:374-386.
[34]HE T,ZHANG Z,ZHANG H,et al.Bag of Tricks for ImageClassification with Convolutional Neural Networks[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).2019:558-567.
[35]ROSSLER A,COZZOLINO D,VERDOLIVA L,et al.FaceFo-rensics++:Learning to Detect Manipulated Facial Images[C]//2019 IEEE/CVF International Conference on Computer Vision(ICCV).IEEE,2019.
[36]HARA K,KATAOKA H,SATOH Y.Learning Spatio-Temporal Features with 3D Residual Networks for Action Recognition[C]//2017 IEEE International Conference on Computer Vision Workshops(ICCVW).2017:3154-3160.
[37]TRAN D,WANG H,FEISZLI M,et al.Video ClassificationWith Channel-Separated Convolutional Networks[C]//Procee-dings of the IEEE International Conference on Computer Vision.2019:5551-5560.
[38]HALIASSOS A,VOUGIOUKAS K,PETRIDIS S,et al.LipsDon’t Lie:A Generalisable and Robust Approach to Face Forgery Detection[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).2021:5037-5047.
[39]Captum[CP/OL].https://captum.ai/.
[40]ZHAO T,XU X,XU M,et al.Learning Self-Consistency forDeepfake Detection[C]//2021 IEEE/CVF International Confe-rence on Computer Vision(ICCV).2021:15003-15013.
[41]BONDI L,DANIELE CANNAS E,BESTAGINI P,et al.Trai-ning Strategies and Data Augmentations in CNN-based DeepFake Video Detection[C]//2020 IEEE International Workshop on Information Forensics and Security(WIFS).2020:1-6.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

Comments

Recommended 0

No Suggested Reading articles found!

Deepfake Face Tampering Video Detection Method Based on Non-critical Masks and AttentionMechanism

PDF (PC)

Abstract

Cite this article

share this article

References

Related Articles 5

Metrics

Comments

Recommended 0

[1]	SHI Da, LU Tian-liang, DU Yan-hui, ZHANG Jian-ling, BAO Yu-xuan. Generation Model of Gender-forged Face Image Based on Improved CycleGAN [J]. Computer Science, 2022, 49(2): 31-39.
[2]	LI Ying, BIAN Shan, WANG Chun-tao, HUANG Qiong. Detection of Deepfakes Based on Dual-stream Network [J]. Computer Science, 2022, 49(11A): 220100106-9.
[3]	BAO Yu-xuan, LU Tian-liang, DU Yan-hui, SHI Da. Deepfake Videos Detection Method Based on i_ResNet34 Model and Data Augmentation [J]. Computer Science, 2021, 48(7): 77-85.
[4]	XING Hao, LI Ming. Deepfake Video Detection Based on 3D Convolutional Neural Networks [J]. Computer Science, 2021, 48(7): 86-92.
[5]	BAO Yu-xuan, LU Tian-liang, DU Yan-hui. Overview of Deepfake Video Detection Technology [J]. Computer Science, 2020, 47(9): 283-292.