Computer Science ›› 2021, Vol. 48 ›› Issue (7): 77-85.doi: 10.11896/jsjkx.210300258

Special Issue: Artificial Intelligence Security

• Artificial Intelligence Security • Previous Articles     Next Articles

Deepfake Videos Detection Method Based on i_ResNet34 Model and Data Augmentation

BAO Yu-xuan, LU Tian-liang, DU Yan-hui, SHI Da   

  1. College of Information and Cyber Security,People’s Public Security University of China,Beijing 100038,China
  • Received:2021-03-25 Revised:2021-04-29 Online:2021-07-15 Published:2021-07-02
  • About author:BAO Yu-xuan,born in 1997,master.His main research interests include cyber security and artificial intelligence.(
    LU Tian-liang,born in 1985,Ph.D,associate professor,is a member of China Computer Federation.His main research interests include cyber security and artificial intelligence.
  • Supported by:
    National Key R&D Program of China(2017YFB0802804) and 2020 Fundamental Research Funds for the Central Universities of PPSUC(2020JKF101).

Abstract: Existing Deepfake videos detection methods are weak in extracting facial feature.Therefore,this paper proposes an improved ResNet(i_ResNet34) model and three data augmentation methods based on information dropping.Firstly,the ResNet is optimized by using the group convolution to replace the ordinary convolution to extract more sufficient facial features without increasing model parameters.Then,max pooling layer is used to the down sampling in the shortcut branch of the dashed residual structure of the model whichis improved,so that loss of facial feature information decreases in video frames.Then,the channel attention layer is introduced after the convolution layer to increase the weight of the channel which extracts the key features and improves the channel correlation of the feature map.Finally,the i_ResNet34 model is implemented to train the original dataset and the expanded dataset with three data augmentation methods based on information dropping,achieving 99.33% and 98.67% detection accuracy on FaceSwap and Deepfakes datasets of FaceForensicans++ respectively,superior to the existing mainstream algorithms,thus verifying the effectiveness of the proposed method.

Key words: Artificial intelligence security, Data augmentation, Deep learning, Deepfake, Feature extraction, Residual network

CLC Number: 

  • TP309
[1]BBC Bitesize.“Deepfakes:What are They and Why Would IMake One?” [OL].
[2]BAO Y X,LU T L,DU Y H.Overview of Deepfake Video Detection Technology[J].Computer Science,2020,47(9):283-292.
[3]KOOPMAN M,RODRIGUEZ A M,GERADTS Z.Detection of Deepfake Video Manipulation[C]//The 20th Irish Machine Vision and Image Processing Conference (IMVIP).2018:133-136.
[4]LI J C,LIU B B,HU Y J,et al.Deepfake Video Detection Based on Consistency of Illumination Direction[J].Journal of Nanjing University of Aeronautics & Astronautics,2020,52(5):760-767.
[5]MATERN F,RIESS C,STAMMINGER M.Exploiting Visual Artifacts to Expose Deepfakes and Face Manipulations[C]//Proceedings of 2019 IEEE Winter A pplications of Computer Vision Workshops (WACVW).IEEE,2019:83-92.
[6]YANG X,LI Y,LYU S.Exposing Deepfakes Using Inconsistent Head Poses[C]//Proceedings of 2019 IEEE International Conference on Acoustics,Speech and Signal Processing (ICASSP).IEEE,2019:8261-8265.
[7]DURALL R,KEUPER M,PFREUNDT F J,et al.Unmasking deepfakes with simple features[J].arXiv:1911.00686,2019.
[8]RAHMOUNI N,NOZICK V,YAMAGISHI J,et al.Distingui-shing computer graphics from natural images using convolution neural networks[C]//IEEE Workshop on Information Forensics and Security.2017:1-6.
[9]AFCHAR D,NOZICK V,YAMAGISHI al.Mesonet:acompact facial video forgery detection network[C]//IEEE International Workshop on Information Forensics and Security (WIFS’18).2018:1-7.
[10]ZHOU P,HAN X,MORARIU V I,et al.Learning Rich Features for Image Manipulation Detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:1053-1061.
[11]NGUYEN H H,YAMAGISHI J,ECHIZEN I.Capsule-forensics:Using Capsule Networks to Detect Forged Images and Vi-deos[C]//Proceedings of 2019 IEEE International Conference on Acoustics,Speech and Signal Processing (ICASSP).IEEE,2019:2307-2311.
[12]WU X,JIA S J.Face swapping detection based on multi-channel attention mechanism[J/OL].Computer Engineering:
[13]HU Y J,GAO Y F,LIU B B,et al.Deepfake Videos Detection Based on Image Segmentation withDeep Neural Networks[J].Journal of Electronics & Information Technology,2021,43(1):162-170.
[14]SABIR E,CHENG J,JAISWAL A,et al.Recurrent Convolutional Strategies for Face Manipulation Detection in videos[J].Interfaces (GUI),2019,3:1.
[15]LI Y,CHANG M C,LYU S.In Ictu Oculi:Exposing AI Created Fake Videos by Detecting Eye Blinking[C]//2018 IEEE International Workshop on Information Forensics and Security (WIFS).IEEE,2018:1-7.
[16]AMERINI I,GALTERI L,CALDELLI R,et al.Deepfake Video Detection through Optical Flow based CNN[C]//Proceedings of the IEEE International Conference on Computer Vision Workshops.2019:1205-1207.
[17]ZHENG B W,XIA H W,CHEN R D,et al.Exposing DeepFake Videos Based Convolutional LSTM Network[J/OL].Laser & Optoelectronics Progress.
[18]ZHANG Y X,LI G,CAO Y,et al.A Method for Detecting Human-face-tampered Videos based on Interframe Difference[J].Journal of Cyber Security,2020,5(2):49-72.
[19]DENG J,GUO J,ZHOU Y,et al.Retinaface:Single-stage dense face localisation in the wild[J].arXiv:1905.00641,2019.
[20]ZHONG Z,ZHENG L,KANG G,et al.Random erasing dataaugmentation[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2020,34(7):13001-13008.
[21]CHEN P,LIU S,ZHAO H,et al.Gridmask data augmentation[J].arXiv:2001.04086,2020.
[22]HE K M,ZHANG X Y,RENS Q,et al.Deep residual learningfor image recognition[C]//Proceedings of 2016 IEEE Confe-rence on Computer Vision and Pattern Recognition.Las Vegas,USA:IEEE,2016:770-778.
[23]XIE S,GIRSHICK R,DOLLÁR P,et al.Aggregated residualtransformations for deep neural networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:1492-1500.
[24]HU J,SHEN L,SUN G.Squeeze-and-excitation networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:7132-7141.
[25]ROSSLER A,COZZOLINO D,VERDOLIVA L,et al.Face-forensics++:Learning to detect manipulated facial images[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2019:1-11.
[26]CHOLLET F.Xception:Deep Learning with Depthwise Separable Convolutions[C]//IEEE Conference on Computer Vision and Pattern Recognition.2017.
[1] RAO Zhi-shuang, JIA Zhen, ZHANG Fan, LI Tian-rui. Key-Value Relational Memory Networks for Question Answering over Knowledge Graph [J]. Computer Science, 2022, 49(9): 202-207.
[2] TANG Ling-tao, WANG Di, ZHANG Lu-fei, LIU Sheng-yun. Federated Learning Scheme Based on Secure Multi-party Computation and Differential Privacy [J]. Computer Science, 2022, 49(9): 297-305.
[3] XU Yong-xin, ZHAO Jun-feng, WANG Ya-sha, XIE Bing, YANG Kai. Temporal Knowledge Graph Representation Learning [J]. Computer Science, 2022, 49(9): 162-171.
[4] WANG Jian, PENG Yu-qi, ZHAO Yu-fei, YANG Jian. Survey of Social Network Public Opinion Information Extraction Based on Deep Learning [J]. Computer Science, 2022, 49(8): 279-293.
[5] WANG Xin-tong, WANG Xuan, SUN Zhi-xin. Network Traffic Anomaly Detection Method Based on Multi-scale Memory Residual Network [J]. Computer Science, 2022, 49(8): 314-322.
[6] HAO Zhi-rong, CHEN Long, HUANG Jia-cheng. Class Discriminative Universal Adversarial Attack for Text Classification [J]. Computer Science, 2022, 49(8): 323-329.
[7] JIANG Meng-han, LI Shao-mei, ZHENG Hong-hao, ZHANG Jian-peng. Rumor Detection Model Based on Improved Position Embedding [J]. Computer Science, 2022, 49(8): 330-335.
[8] SUN Qi, JI Gen-lin, ZHANG Jie. Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection [J]. Computer Science, 2022, 49(8): 172-177.
[9] ZHANG Yuan, KANG Le, GONG Zhao-hui, ZHANG Zhi-hong. Related Transaction Behavior Detection in Futures Market Based on Bi-LSTM [J]. Computer Science, 2022, 49(7): 31-39.
[10] YANG Bing-xin, GUO Yan-rong, HAO Shi-jie, Hong Ri-chang. Application of Graph Neural Network Based on Data Augmentation and Model Ensemble in Depression Recognition [J]. Computer Science, 2022, 49(7): 57-63.
[11] HU Yan-yu, ZHAO Long, DONG Xiang-jun. Two-stage Deep Feature Selection Extraction Algorithm for Cancer Classification [J]. Computer Science, 2022, 49(7): 73-78.
[12] ZENG Zhi-xian, CAO Jian-jun, WENG Nian-feng, JIANG Guo-quan, XU Bin. Fine-grained Semantic Association Video-Text Cross-modal Entity Resolution Based on Attention Mechanism [J]. Computer Science, 2022, 49(7): 106-112.
[13] CHENG Cheng, JIANG Ai-lian. Real-time Semantic Segmentation Method Based on Multi-path Feature Extraction [J]. Computer Science, 2022, 49(7): 120-126.
[14] HOU Yu-tao, ABULIZI Abudukelimu, ABUDUKELIMU Halidanmu. Advances in Chinese Pre-training Models [J]. Computer Science, 2022, 49(7): 148-163.
[15] ZHOU Hui, SHI Hao-chen, TU Yao-feng, HUANG Sheng-jun. Robust Deep Neural Network Learning Based on Active Sampling [J]. Computer Science, 2022, 49(7): 164-169.
Full text



No Suggested Reading articles found!