深度伪造视频检测技术综述

doi:10.11896/jsjkx.200400130

Abstract

Abstract: The abuse of deepfake brings potential threats to the country,society and individuals.Firstly,this paper introduces the concept and current trend of deepfake,analyzes the generation principle and models of deepfake videos based on generative adversarial networks,and introduces the video data processing algorithms and the mainstream deepfake datasets.Secondly,this paper summarizes the detection methods based on the tampering features in video frames.Aiming at the detection of visual artifacts and facial noise features in deepfake video frames,the classification algorithms and models related to machine learning and deep learning are introduced.Then,specific to inconsistency of time-space state between deepfake video frames,the relevant time series algorithms and detection methods are introduced.Then,the tamper-proof public mechanism based on blockchain tracing and information security methods such as digital watermark and video fingerprinting are introduced as supplementary detection means.Finally,the future research direction of deepfake video detection technology is summarized.

Key words: Deep learning, Deepfake, Feature extraction, Multimedia forensics, Video frame

CLC Number:

TP309

BAO Yu-xuan, LU Tian-liang, DU Yan-hui. Overview of Deepfake Video Detection Technology[J].Computer Science, 2020, 47(9): 283-292.

References

[1] LONG K,MA Y,ZHU Q C.How Will Deepfake Technology Influence National Security:Emerging Challenges and Policy Implications [J].China Information Security,2019(10):21-34.
[2] CHENG X Y,XIE L,ZHU J X,et al.Review of Generative Adversarial Network[J].Computer Science,2019,46(3):74-81.
[3] 深圳英鹏信息技术股份有限公司.向Deepfake宣战！[EB/OL].(2020-01-14)[2020-04-05].https://baijiahao.baidu.com/s?id=1655671833927886540&wfr=spider&for=pc.
[4] CHESNEY R,CITRON D.Deepfakes and the New Disinformation War:The Coming Age of Post-truth Geopolitics[J].Foreign Aff.,2019,98:147.
[5] NEWS ARTICLE.Code of Practice on Disinformation[EB/OL].(2018-09-26)[2020-04-05].https://ec.europa.eu/digital-single-market/en/news/code-practice-disinformation.
[6] 国家互联网信息办公室等三部门规范网络音视频信息服务[J].中国广播,2019(12):39.
[7] HINTON G E,SALAKHUTDINOV R.Reducing the Dimen-sionality of Data with Neural Networks [J].Science,2006,313(5786):504-507.
[8] NGUYEN T T,NGUYEN C M,NGUYEN D T,et al.DeepLearning for Deepfakes Creation and Detection[J].arXiv:1909.11573,2019.
[9] GOODFELLOW I,POUGET-ABADIE J,MIRZA M,et al.Gen-erative Adversarial Nets[C]//Advances in Neural Information Processing Systems.2014:2672-2680.
[10] RADFORD A,METZ L,CHINTALA S.Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks[J].arXiv:1511.06434,2015.
[11] ARJOVSKY M,CHINTALA S,BOTTOU L.Wasserstein GAN[J].arXiv:1701.07875,2017.
[12] ZHU J Y,PARK T,ISOLA P,et al.Unpaired Image-to-image Translation Using Cycle-consistent Adversarial Networks[C]//Proceedings of the IEEE International Conference on Computer Vision.2017:2223-2232.
[13] WANG C,XU C,YAO X,et al.Evolutionary Generative Adversarial Networks[J].IEEE Transactions on Evolutionary Computation,2019,23(6):921-934.
[14] KARRAS T,LAINE S,AILA T.A Style-based Generator Architecture for Generative Adversarial Networks[C]//Procee-dings of the IEEE Conference on Computer Vision and Pattern Recognition.2019:4401-4410.
[15] Faceswap.Faceswap is the leading free and Open Source multi-platform Deepfakes software [EB/OL].(2019-08-15)[2020-04-23].https://faceswap.dev/.
[16] Shaoanlu.Faceswap-GAN[EB/OL].(2019-10-04)[2020-04-23].https://github.com/shaoanlu/faceswap-GAN.
[17] dfaker.DFaker[EB/OL].(2018-02-24)[2020-04-23].https://github.com/dfaker/df.
[18] SU M L.ZAO Privacy Protection Saves "big hole"[J].Computers & Networks,2019,45(17):8-10.
[19] ROSSLER A,COZZOLINO D,VERDOLIVA L,et al.Faceforensics++:Learning to Detect Manipulated Facial Images[C]//Proceedings of the IEEE International Conference on Computer Vision.2019:1-11.
[20] KORSHUNOV P,MARCEL S.Deepfakes:a New Threat toFace Recognition? Assessment and Detection[J].arXiv:1812.08685,2018.
[21] LI Y,YANG X,SUN P,et al.Celeb-df:A new Dataset for Deepfake Forensics[J].arXiv:1909.12962,2019.
[22] WANG X,DING H,CHEN H.A Shot Clustering Based App-roach for Scene Segmentation[J].Journal of Image and Gra-phics,2007,12(1):2127-2131.
[23] ZOLFAGHARI M,SINGH K,BROX T.Eco:Efficient Convolutional Network for Online Video Understanding[C]//Procee-dings of the European Confe-rence on Computer Vision (ECCV).2018:695-712.
[24] YU X D,WANG L,TIAN Q,et al.Multilevel Video Representation with Application to Keyframe Extraction[C]//Proceedings of the 10th International Multimedia Modelling Conference.IEEE,2004:117-123.
[25] YIN Y,JIANG H N.Key Frame Extraction Based on Clustering of Optimizing Initial Centers[J].Computer Engineering and Applications,2007(21):165-167.
[26] EJAZ N,BAIK S W,MAJEED H,et al.Multi-scale Contrastand Relative Motion-based Key Frame Extraction[J].EURASIP Journal on Image and Video Processing,2018,2018(1):40.
[27] WOLF W.Key Frame Selection by Motion Analysis[C]//Proceedings of the 1996 IEEE International Conference on Acoustics,Speech,and Signal Processing Conference.IEEE,1996:1228-1231.
[28] COOTES T F,TAYLOR C J,LANITIS A.Active Shape Models:Evaluation of a Multi-Resolution Method for Improving Image Search[C]//BMVC.1994:327-336.
[29] COOTES T F,EDWARDS G J,TAYLOR C J.Active Appearance Models[C]//European Conference on Computer Vision.Berlin:Springer,1998:484-498.
[30] KAZEMI V,SULLIVAN J.One Millisecond Face Alignmentwith an Ensemble of Regression Trees[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2014:1867-1874.
[31] CAO X,WEI Y,WEN F,et al.Face Alignment by ExplicitShape Regression[J].International Journal of Computer Vision,2014,107(2):177-190.
[32] BURGOS-ARTIZZU X P,PERONA P,DOLLÁR P.RobustFace Landmark Estimation under Occlusion[C]//Proceedings of the IEEE International Conference on Computer Vision.2013:1513-1520.
[33] SUN Y,WANG X,TANG X.Deep Convolutional Network Cascade for Facial Point Detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2013:3476-3483.
[34] ZHOU E,FAN H,CAO Z,et al.Extensive Facial Landmark Localization with Coarse-to-fine Convolutional Network Cascade[C]//Proceedings of the IEEE International Conference on Computer Vision Workshops.2013:386-391.
[35] MATERN F,RIESS C,STAMMINGER M.Exploiting Visual Artifacts to Expose Deepfakes and Face Manipulations[C]//Proceedings of 2019 IEEE Winter Applications of Computer Vision Workshops (WACVW).IEEE,2019:83-92.
[36] YANG X,LI Y,LYU S.Exposing Deepfakes Using Inconsistent Head Poses[C]//Proceedings of 2019 IEEE International Conference on Acoustics,Speech and Signal Processing (ICASSP).IEEE,2019:8261-8265.
[37] YANG X,LI Y,QI H,et al.Exposing Gan-synthesized Faces Using Landmark Locations[C]//Proceedings of the ACM Workshop on Information Hiding and Multimedia Security.2019:113-118.
[38] WADHWA N,WU H Y,DAVIS A,et al.Eulerian Video Magnification and Analysis[J].Communications of the ACM,2016,60(1):87-95.
[39] LI Y,LYU S.Exposing Deepfake Videos by Detecting FaceWarping Artifacts[J].arXiv:1811.00656,2018.
[40] SIMONYAN K,ZISSERMAN A.Very Deep Convolutional Networks for Large-scale Image Recognition[J].arXiv:1409.1556,2014.
[41] HE K,ZHANG X,REN S,et al.Deep Residual Learning forImage Recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:770-778.
[42] ZHOU P,HAN X,MORARIU V I,et al.Learning Rich Fea-tures for Image Manipulation Detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:1053-1061.
[43] AFCHAR D,NOZICK V,YAMAGISHI J,et al.Mesonet:ACompact Facial Video Forgery Detection Network[C]//2018 IEEE International Workshop on Information Forensics and Security (WIFS).IEEE,2018:1-7.
[44] ZHANG P,ZOU F,WU Z,et al.FeatherNets:ConvolutionalNeural Networks as Light as Feather for Face Anti-spoofing[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops.2019:1574-1583.
[45] BAYAR B,STAMM M C.Constrained Convolutional NeuralNetworks:A New Approach Towards General Purpose Image Manipulation Detection[J].IEEE Transactions on Information Forensics and Security,2018,13(11):2691-2706.
[46] NGUYEN H H,YAMAGISHI J,ECHIZEN I.Capsule-forensics:Using Capsule Networks to Detect Forged Images and Videos[C]//Proceedings of 2019 IEEE International Conference on Acoustics,Speech and Signal Processing (ICASSP).IEEE,2019:2307-2311.
[47] GÜERA D,DELP J.Deepfake Video Detection Using Recurrent Neural Networks[C]//2018 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).IEEE,2018:1-6.
[48] HOCHREITER S,SCHMIDHUBER J.Long Short-term Memory[J].Neural Computation,1997,9(8):1735-1780.
[49] SABIR E,CHENG J,JAISWAL A,et al.Recurrent Convolutional Strategies for Face Manipulation Detection in videos[J].Interfaces (GUI),2019,3:1.
[50] LI Y,CHANG M C,LYU S.In Ictu Oculi:Exposing AI Created Fake Videos by Detecting Eye Blinking[C]//2018 IEEE International Workshop on Information Forensics and Security (WIFS).IEEE,2018:1-7.
[51] AMERINI I,GALTERI L,CALDELLI R,et al.Deepfake Video Detection through Optical Flow based CNN[C]//Proceedings of the IEEE International Conference on Computer Vision Workshops.2019:1205-1207.
[52] HASAN H R,SALAH K.Combating Deepfake Videos Using Blockchain and Smart Contracts[J].IEEE Access,2019,7:41596-41606.
[53] KOOPMAN M,RODRIGUEZ A M,GERADTS Z.Detection of Deepfake Video Manipulation[C]//The 20th Irish Machine Vision and Image Processing Conference (IMVIP).2018:133-136.
[54] LEE S,YOO C D.Video Fingerprinting Based on Centroids of Gradient Orientations[C]//Proceedings of 2006 IEEE International Conference on AcousticsSpeech and Signal Processing.IEEE,2006,2:1-4.
[55] TANG W,WO Y,HAN G.Geometrically Robust Video Hashing Based on ST-PCT for Video Copy Detection[J].Multimedia Tools and Applications,2019,78(15):21999-22022.
[56] LI J,ZHANG H,WAN W,et al.Two-class 3D-CNN Classifiers Combination for Video Copy Detection[J].Multimedia Tools and Applications,2018,3:1-13.
[57] MA C,GU Y,GONG C,et al.Unsupervised Video Hashing via Deep Neural Network[J].Neural Processing Letters,2018,47(3):877-890.
[58] CHEN P,LIANG T,DAI J,et al.Forged Facial Video Detection Based on Global Temporal and Local Spatial Feature[J].Journal of Cyber Security,2020,5(2):73-83.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

Comments

Recommended 0

No Suggested Reading articles found!

Overview of Deepfake Video Detection Technology

PDF (PC)

Abstract

Cite this article

share this article

References

Related Articles 15

Metrics

Comments

Recommended 0

[1]	RAO Zhi-shuang, JIA Zhen, ZHANG Fan, LI Tian-rui. Key-Value Relational Memory Networks for Question Answering over Knowledge Graph [J]. Computer Science, 2022, 49(9): 202-207.
[2]	TANG Ling-tao, WANG Di, ZHANG Lu-fei, LIU Sheng-yun. Federated Learning Scheme Based on Secure Multi-party Computation and Differential Privacy [J]. Computer Science, 2022, 49(9): 297-305.
[3]	XU Yong-xin, ZHAO Jun-feng, WANG Ya-sha, XIE Bing, YANG Kai. Temporal Knowledge Graph Representation Learning [J]. Computer Science, 2022, 49(9): 162-171.
[4]	WANG Jian, PENG Yu-qi, ZHAO Yu-fei, YANG Jian. Survey of Social Network Public Opinion Information Extraction Based on Deep Learning [J]. Computer Science, 2022, 49(8): 279-293.
[5]	HAO Zhi-rong, CHEN Long, HUANG Jia-cheng. Class Discriminative Universal Adversarial Attack for Text Classification [J]. Computer Science, 2022, 49(8): 323-329.
[6]	JIANG Meng-han, LI Shao-mei, ZHENG Hong-hao, ZHANG Jian-peng. Rumor Detection Model Based on Improved Position Embedding [J]. Computer Science, 2022, 49(8): 330-335.
[7]	SUN Qi, JI Gen-lin, ZHANG Jie. Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection [J]. Computer Science, 2022, 49(8): 172-177.
[8]	ZHANG Yuan, KANG Le, GONG Zhao-hui, ZHANG Zhi-hong. Related Transaction Behavior Detection in Futures Market Based on Bi-LSTM [J]. Computer Science, 2022, 49(7): 31-39.
[9]	HU Yan-yu, ZHAO Long, DONG Xiang-jun. Two-stage Deep Feature Selection Extraction Algorithm for Cancer Classification [J]. Computer Science, 2022, 49(7): 73-78.
[10]	ZENG Zhi-xian, CAO Jian-jun, WENG Nian-feng, JIANG Guo-quan, XU Bin. Fine-grained Semantic Association Video-Text Cross-modal Entity Resolution Based on Attention Mechanism [J]. Computer Science, 2022, 49(7): 106-112.
[11]	CHENG Cheng, JIANG Ai-lian. Real-time Semantic Segmentation Method Based on Multi-path Feature Extraction [J]. Computer Science, 2022, 49(7): 120-126.
[12]	HOU Yu-tao, ABULIZI Abudukelimu, ABUDUKELIMU Halidanmu. Advances in Chinese Pre-training Models [J]. Computer Science, 2022, 49(7): 148-163.
[13]	ZHOU Hui, SHI Hao-chen, TU Yao-feng, HUANG Sheng-jun. Robust Deep Neural Network Learning Based on Active Sampling [J]. Computer Science, 2022, 49(7): 164-169.
[14]	SU Dan-ning, CAO Gui-tao, WANG Yan-nan, WANG Hong, REN He. Survey of Deep Learning for Radar Emitter Identification Based on Small Sample [J]. Computer Science, 2022, 49(7): 226-235.
[15]	LIU Wei-ye, LU Hui-min, LI Yu-peng, MA Ning. Survey on Finger Vein Recognition Research [J]. Computer Science, 2022, 49(6A): 1-11.