Computer Science

Computer Science ›› 2020, Vol. 47 ›› Issue (9): 283-292.doi: 10.11896/jsjkx.200400130

• Information Security • Previous Articles     Next Articles

Overview of Deepfake Video Detection Technology

BAO Yu-xuan, LU Tian-liang, DU Yan-hui   

  1. College of Police Information Engineering and Network Security,People’s Public Security University of China,Beijing 100038,China
  • Received:2020-04-28 Published:2020-09-10
  • About author:BAO Yu-xuan,born in 1997,master.His main research interests include cyber security and artificial intelligence.
    LU Tian-liang,born in 1985,Ph.D,associate professor,is a member of China Computer Federation.His main research interests include cyber security and artificial intelligence.
  • Supported by:
    National Key R&D Program of China (20190178) and Fundamental Research Funds for the Central Universities of PPSUC (2020JKF101).

PDF (PC)

521

Abstract: The abuse of deepfake brings potential threats to the country,society and individuals.Firstly,this paper introduces the concept and current trend of deepfake,analyzes the generation principle and models of deepfake videos based on generative adversarial networks,and introduces the video data processing algorithms and the mainstream deepfake datasets.Secondly,this paper summarizes the detection methods based on the tampering features in video frames.Aiming at the detection of visual artifacts and facial noise features in deepfake video frames,the classification algorithms and models related to machine learning and deep learning are introduced.Then,specific to inconsistency of time-space state between deepfake video frames,the relevant time series algorithms and detection methods are introduced.Then,the tamper-proof public mechanism based on blockchain tracing and information security methods such as digital watermark and video fingerprinting are introduced as supplementary detection means.Finally,the future research direction of deepfake video detection technology is summarized.

Key words: Deepfake, Deep learning, Feature extraction, Video frame, Multimedia forensics

CLC Number: 

  • TP309
[1] LONG K,MA Y,ZHU Q C.How Will Deepfake Technology Influence National Security:Emerging Challenges and Policy Implications [J].China Information Security,2019(10):21-34.
[2] CHENG X Y,XIE L,ZHU J X,et al.Review of Generative Adversarial Network[J].Computer Science,2019,46(3):74-81.
[3] 深圳英鹏信息技术股份有限公司.向Deepfake宣战！[EB/OL].(2020-01-14)[2020-04-05].https://baijiahao.baidu.com/s?id=1655671833927886540&wfr=spider&for=pc.
[4] CHESNEY R,CITRON D.Deepfakes and the New Disinformation War:The Coming Age of Post-truth Geopolitics[J].Foreign Aff.,2019,98:147.
[5] NEWS ARTICLE.Code of Practice on Disinformation[EB/OL].(2018-09-26)[2020-04-05].https://ec.europa.eu/digital-single-market/en/news/code-practice-disinformation.
[6] 国家互联网信息办公室等三部门规范网络音视频信息服务[J].中国广播,2019(12):39.
[7] HINTON G E,SALAKHUTDINOV R.Reducing the Dimen-sionality of Data with Neural Networks [J].Science,2006,313(5786):504-507.
[8] NGUYEN T T,NGUYEN C M,NGUYEN D T,et al.DeepLearning for Deepfakes Creation and Detection[J].arXiv:1909.11573,2019.
[9] GOODFELLOW I,POUGET-ABADIE J,MIRZA M,et al.Gen-erative Adversarial Nets[C]//Advances in Neural Information Processing Systems.2014:2672-2680.
[10] RADFORD A,METZ L,CHINTALA S.Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks[J].arXiv:1511.06434,2015.
[11] ARJOVSKY M,CHINTALA S,BOTTOU L.Wasserstein GAN[J].arXiv:1701.07875,2017.
[12] ZHU J Y,PARK T,ISOLA P,et al.Unpaired Image-to-image Translation Using Cycle-consistent Adversarial Networks[C]//Proceedings of the IEEE International Conference on Computer Vision.2017:2223-2232.
[13] WANG C,XU C,YAO X,et al.Evolutionary Generative Adversarial Networks[J].IEEE Transactions on Evolutionary Computation,2019,23(6):921-934.
[14] KARRAS T,LAINE S,AILA T.A Style-based Generator Architecture for Generative Adversarial Networks[C]//Procee-dings of the IEEE Conference on Computer Vision and Pattern Recognition.2019:4401-4410.
[15] Faceswap.Faceswap is the leading free and Open Source multi-platform Deepfakes software [EB/OL].(2019-08-15)[2020-04-23].https://faceswap.dev/.
[16] Shaoanlu.Faceswap-GAN[EB/OL].(2019-10-04)[2020-04-23].https://github.com/shaoanlu/faceswap-GAN.
[17] dfaker.DFaker[EB/OL].(2018-02-24)[2020-04-23].https://github.com/dfaker/df.
[18] SU M L.ZAO Privacy Protection Saves "big hole"[J].Computers & Networks,2019,45(17):8-10.
[19] ROSSLER A,COZZOLINO D,VERDOLIVA L,et al.Faceforensics++:Learning to Detect Manipulated Facial Images[C]//Proceedings of the IEEE International Conference on Computer Vision.2019:1-11.
[20] KORSHUNOV P,MARCEL S.Deepfakes:a New Threat toFace Recognition? Assessment and Detection[J].arXiv:1812.08685,2018.
[21] LI Y,YANG X,SUN P,et al.Celeb-df:A new Dataset for Deepfake Forensics[J].arXiv:1909.12962,2019.
[22] WANG X,DING H,CHEN H.A Shot Clustering Based App-roach for Scene Segmentation[J].Journal of Image and Gra-phics,2007,12(1):2127-2131.
[23] ZOLFAGHARI M,SINGH K,BROX T.Eco:Efficient Convolutional Network for Online Video Understanding[C]//Procee-dings of the European Confe-rence on Computer Vision (ECCV).2018:695-712.
[24] YU X D,WANG L,TIAN Q,et al.Multilevel Video Representation with Application to Keyframe Extraction[C]//Proceedings of the 10th International Multimedia Modelling Conference.IEEE,2004:117-123.
[25] YIN Y,JIANG H N.Key Frame Extraction Based on Clustering of Optimizing Initial Centers[J].Computer Engineering and Applications,2007(21):165-167.
[26] EJAZ N,BAIK S W,MAJEED H,et al.Multi-scale Contrastand Relative Motion-based Key Frame Extraction[J].EURASIP Journal on Image and Video Processing,2018,2018(1):40.
[27] WOLF W.Key Frame Selection by Motion Analysis[C]//Proceedings of the 1996 IEEE International Conference on Acoustics,Speech,and Signal Processing Conference.IEEE,1996:1228-1231.
[28] COOTES T F,TAYLOR C J,LANITIS A.Active Shape Models:Evaluation of a Multi-Resolution Method for Improving Image Search[C]//BMVC.1994:327-336.
[29] COOTES T F,EDWARDS G J,TAYLOR C J.Active Appearance Models[C]//European Conference on Computer Vision.Berlin:Springer,1998:484-498.
[30] KAZEMI V,SULLIVAN J.One Millisecond Face Alignmentwith an Ensemble of Regression Trees[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2014:1867-1874.
[31] CAO X,WEI Y,WEN F,et al.Face Alignment by ExplicitShape Regression[J].International Journal of Computer Vision,2014,107(2):177-190.
[32] BURGOS-ARTIZZU X P,PERONA P,DOLLÁR P.RobustFace Landmark Estimation under Occlusion[C]//Proceedings of the IEEE International Conference on Computer Vision.2013:1513-1520.
[33] SUN Y,WANG X,TANG X.Deep Convolutional Network Cascade for Facial Point Detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2013:3476-3483.
[34] ZHOU E,FAN H,CAO Z,et al.Extensive Facial Landmark Localization with Coarse-to-fine Convolutional Network Cascade[C]//Proceedings of the IEEE International Conference on Computer Vision Workshops.2013:386-391.
[35] MATERN F,RIESS C,STAMMINGER M.Exploiting Visual Artifacts to Expose Deepfakes and Face Manipulations[C]//Proceedings of 2019 IEEE Winter Applications of Computer Vision Workshops (WACVW).IEEE,2019:83-92.
[36] YANG X,LI Y,LYU S.Exposing Deepfakes Using Inconsistent Head Poses[C]//Proceedings of 2019 IEEE International Conference on Acoustics,Speech and Signal Processing (ICASSP).IEEE,2019:8261-8265.
[37] YANG X,LI Y,QI H,et al.Exposing Gan-synthesized Faces Using Landmark Locations[C]//Proceedings of the ACM Workshop on Information Hiding and Multimedia Security.2019:113-118.
[38] WADHWA N,WU H Y,DAVIS A,et al.Eulerian Video Magnification and Analysis[J].Communications of the ACM,2016,60(1):87-95.
[39] LI Y,LYU S.Exposing Deepfake Videos by Detecting FaceWarping Artifacts[J].arXiv:1811.00656,2018.
[40] SIMONYAN K,ZISSERMAN A.Very Deep Convolutional Networks for Large-scale Image Recognition[J].arXiv:1409.1556,2014.
[41] HE K,ZHANG X,REN S,et al.Deep Residual Learning forImage Recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:770-778.
[42] ZHOU P,HAN X,MORARIU V I,et al.Learning Rich Fea-tures for Image Manipulation Detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:1053-1061.
[43] AFCHAR D,NOZICK V,YAMAGISHI J,et al.Mesonet:ACompact Facial Video Forgery Detection Network[C]//2018 IEEE International Workshop on Information Forensics and Security (WIFS).IEEE,2018:1-7.
[44] ZHANG P,ZOU F,WU Z,et al.FeatherNets:ConvolutionalNeural Networks as Light as Feather for Face Anti-spoofing[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops.2019:1574-1583.
[45] BAYAR B,STAMM M C.Constrained Convolutional NeuralNetworks:A New Approach Towards General Purpose Image Manipulation Detection[J].IEEE Transactions on Information Forensics and Security,2018,13(11):2691-2706.
[46] NGUYEN H H,YAMAGISHI J,ECHIZEN I.Capsule-forensics:Using Capsule Networks to Detect Forged Images and Videos[C]//Proceedings of 2019 IEEE International Conference on Acoustics,Speech and Signal Processing (ICASSP).IEEE,2019:2307-2311.
[47] GÜERA D,DELP J.Deepfake Video Detection Using Recurrent Neural Networks[C]//2018 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).IEEE,2018:1-6.
[48] HOCHREITER S,SCHMIDHUBER J.Long Short-term Memory[J].Neural Computation,1997,9(8):1735-1780.
[49] SABIR E,CHENG J,JAISWAL A,et al.Recurrent Convolutional Strategies for Face Manipulation Detection in videos[J].Interfaces (GUI),2019,3:1.
[50] LI Y,CHANG M C,LYU S.In Ictu Oculi:Exposing AI Created Fake Videos by Detecting Eye Blinking[C]//2018 IEEE International Workshop on Information Forensics and Security (WIFS).IEEE,2018:1-7.
[51] AMERINI I,GALTERI L,CALDELLI R,et al.Deepfake Video Detection through Optical Flow based CNN[C]//Proceedings of the IEEE International Conference on Computer Vision Workshops.2019:1205-1207.
[52] HASAN H R,SALAH K.Combating Deepfake Videos Using Blockchain and Smart Contracts[J].IEEE Access,2019,7:41596-41606.
[53] KOOPMAN M,RODRIGUEZ A M,GERADTS Z.Detection of Deepfake Video Manipulation[C]//The 20th Irish Machine Vision and Image Processing Conference (IMVIP).2018:133-136.
[54] LEE S,YOO C D.Video Fingerprinting Based on Centroids of Gradient Orientations[C]//Proceedings of 2006 IEEE International Conference on AcousticsSpeech and Signal Processing.IEEE,2006,2:1-4.
[55] TANG W,WO Y,HAN G.Geometrically Robust Video Hashing Based on ST-PCT for Video Copy Detection[J].Multimedia Tools and Applications,2019,78(15):21999-22022.
[56] LI J,ZHANG H,WAN W,et al.Two-class 3D-CNN Classifiers Combination for Video Copy Detection[J].Multimedia Tools and Applications,2018,3:1-13.
[57] MA C,GU Y,GONG C,et al.Unsupervised Video Hashing via Deep Neural Network[J].Neural Processing Letters,2018,47(3):877-890.
[58] CHEN P,LIANG T,DAI J,et al.Forged Facial Video Detection Based on Global Temporal and Local Spatial Feature[J].Journal of Cyber Security,2020,5(2):73-83.
[1] LIU Yang, JIN Zhong. Fine-grained Image Recognition Method Combining with Non-local and Multi-region Attention Mechanism [J]. Computer Science, 2021, 48(1): 197-203.
[2] WANG Rui-ping, JIA Zhen, LIU Chang, CHEN Ze-wei, LI Tian-rui. Deep Interest Factorization Machine Network Based on DeepFM [J]. Computer Science, 2021, 48(1): 226-232.
[3] YU Wen-jia, DING Shi-fei. Conditional Generative Adversarial Network Based on Self-attention Mechanism [J]. Computer Science, 2021, 48(1): 241-246.
[4] TONG Xin, WANG Bin-jun, WANG Run-zheng, PAN Xiao-qin. Survey on Adversarial Sample of Deep Learning Towards Natural Language Processing [J]. Computer Science, 2021, 48(1): 258-267.
[5] DING Yu, WEI Hao, PAN Zhi-song, LIU Xin. Survey of Network Representation Learning [J]. Computer Science, 2020, 47(9): 52-59.
[6] HE Xin, XU Juan, JIN Ying-ying. Action-related Network:Towards Modeling Complete Changeable Action [J]. Computer Science, 2020, 47(9): 123-128.
[7] YE Ya-nan, CHI Jing, YU Zhi-ping, ZHAN Yu-liand ZHANG Cai-ming. Expression Animation Synthesis Based on Improved CycleGan Model and Region Segmentation [J]. Computer Science, 2020, 47(9): 142-149.
[8] DENG Liang, XU Geng-lin, LI Meng-jie, CHEN Zhang-jin. Fast Face Recognition Based on Deep Learning and Multiple Hash Similarity Weighting [J]. Computer Science, 2020, 47(9): 163-168.
[9] WANG Liang, ZHOU Xin-zhi, YNA Hua. Real-time SIFT Algorithm Based on GPU [J]. Computer Science, 2020, 47(8): 105-111.
[10] LIANG Zheng-you, HE Jing-lin, SUN Yu. Three-dimensional Convolutional Neural Network Evolution Method for Facial Micro-expression Auto-recognition [J]. Computer Science, 2020, 47(8): 227-232.
[11] YUAN Ye, HE Xiao-ge, ZHU Ding-kun, WANG Fu-lee, XIE Hao-ran, WANG Jun, WEI Ming-qiang, GUO Yan-wen. Survey of Visual Image Saliency Detection [J]. Computer Science, 2020, 47(7): 84-91.
[12] WANG Wen-dao, WANG Run-ze, WEI Xin-lei, QI Yun-liang, MA Yi-de. Automatic Recognition of ECG Based on Stacked Bidirectional LSTM [J]. Computer Science, 2020, 47(7): 118-124.
[13] LIU Yan, WEN Jing. Complex Scene Text Detection Based on Attention Mechanism [J]. Computer Science, 2020, 47(7): 135-140.
[14] ZHANG Zhi-yang, ZHANG Feng-li, TAN Qi, WANG Rui-jin. Review of Information Cascade Prediction Methods Based on Deep Learning [J]. Computer Science, 2020, 47(7): 141-153.
[15] JIANG Wen-bin, FU Zhi, PENG Jing, ZHU Jian. 4Bit-based Gradient Compression Method for Distributed Deep Learning System [J]. Computer Science, 2020, 47(7): 220-226.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
[1] LEI Li-hui and WANG Jing. Parallelization of LTL Model Checking Based on Possibility Measure[J]. Computer Science, 2018, 45(4): 71 -75 .
[2] SUN Qi, JIN Yan, HE Kun and XU Ling-xuan. Hybrid Evolutionary Algorithm for Solving Mixed Capacitated General Routing Problem[J]. Computer Science, 2018, 45(4): 76 -82 .
[3] ZHANG Jia-nan and XIAO Ming-yu. Approximation Algorithm for Weighted Mixed Domination Problem[J]. Computer Science, 2018, 45(4): 83 -88 .
[4] WU Jian-hui, HUANG Zhong-xiang, LI Wu, WU Jian-hui, PENG Xin and ZHANG Sheng. Robustness Optimization of Sequence Decision in Urban Road Construction[J]. Computer Science, 2018, 45(4): 89 -93 .
[5] SHI Wen-jun, WU Ji-gang and LUO Yu-chun. Fast and Efficient Scheduling Algorithms for Mobile Cloud Offloading[J]. Computer Science, 2018, 45(4): 94 -99 .
[6] ZHOU Yan-ping and YE Qiao-lin. L1-norm Distance Based Least Squares Twin Support Vector Machine[J]. Computer Science, 2018, 45(4): 100 -105 .
[7] LIU Bo-yi, TANG Xiang-yan and CHENG Jie-ren. Recognition Method for Corn Borer Based on Templates Matching in Muliple Growth Periods[J]. Computer Science, 2018, 45(4): 106 -111 .
[8] YANG Yu-qi, ZHANG Guo-an and JIN Xi-long. Dual-cluster-head Routing Protocol Based on Vehicle Density in VANETs[J]. Computer Science, 2018, 45(4): 126 -130 .
[9] SHI Chao, XIE Zai-peng, LIU Han and LV Xin. Optimization of Container Deployment Strategy Based on Stable Matching[J]. Computer Science, 2018, 45(4): 131 -136 .
[10] HAN Kui-kui, XIE Zai-peng and LV Xin. Fog Computing Task Scheduling Strategy Based on Improved Genetic Algorithm[J]. Computer Science, 2018, 45(4): 137 -142 .
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.