深度伪造视频检测技术综述

doi:10.11896/jsjkx.200400130

摘要/Abstract

摘要： 深度伪造的滥用,给国家、社会和个人带来了潜在威胁。首先,介绍了深度伪造的概念和当前发展趋势,分析了基于生成对抗网络的深度伪造视频的生成原理和模型,并介绍了视频数据处理算法及主流的深度伪造数据集;其次,综述了基于视频帧内篡改特征的检测方法,针对深度伪造视频帧内的视觉伪影、面部噪声特征的检测问题,介绍了相关机器学习、深度学习等分类算法、模型;然后,针对深度伪造视频在帧间时空状态不一致的情形,阐述了相关时间序列算法和检测方法;接着,介绍了作为检测补充手段的基于区块链溯源的防篡改公共机制和数字水印、视频指纹等信息安全方法;最后,总结了深度伪造视频检测技术的未来研究方向。

关键词: 多媒体取证, 深度伪造, 深度学习, 视频帧, 特征提取

Abstract: The abuse of deepfake brings potential threats to the country,society and individuals.Firstly,this paper introduces the concept and current trend of deepfake,analyzes the generation principle and models of deepfake videos based on generative adversarial networks,and introduces the video data processing algorithms and the mainstream deepfake datasets.Secondly,this paper summarizes the detection methods based on the tampering features in video frames.Aiming at the detection of visual artifacts and facial noise features in deepfake video frames,the classification algorithms and models related to machine learning and deep learning are introduced.Then,specific to inconsistency of time-space state between deepfake video frames,the relevant time series algorithms and detection methods are introduced.Then,the tamper-proof public mechanism based on blockchain tracing and information security methods such as digital watermark and video fingerprinting are introduced as supplementary detection means.Finally,the future research direction of deepfake video detection technology is summarized.

Key words: Deep learning, Deepfake, Feature extraction, Multimedia forensics, Video frame

中图分类号:

TP309

暴雨轩, 芦天亮, 杜彦辉. 深度伪造视频检测技术综述[J]. 计算机科学, 2020, 47(9): 283-292. https://doi.org/10.11896/jsjkx.200400130

BAO Yu-xuan, LU Tian-liang, DU Yan-hui. Overview of Deepfake Video Detection Technology[J]. Computer Science, 2020, 47(9): 283-292. https://doi.org/10.11896/jsjkx.200400130

参考文献

[1] LONG K,MA Y,ZHU Q C.How Will Deepfake Technology Influence National Security:Emerging Challenges and Policy Implications [J].China Information Security,2019(10):21-34.
[2] CHENG X Y,XIE L,ZHU J X,et al.Review of Generative Adversarial Network[J].Computer Science,2019,46(3):74-81.
[3] 深圳英鹏信息技术股份有限公司.向Deepfake宣战！[EB/OL].(2020-01-14)[2020-04-05].https://baijiahao.baidu.com/s?id=1655671833927886540&wfr=spider&for=pc.
[4] CHESNEY R,CITRON D.Deepfakes and the New Disinformation War:The Coming Age of Post-truth Geopolitics[J].Foreign Aff.,2019,98:147.
[5] NEWS ARTICLE.Code of Practice on Disinformation[EB/OL].(2018-09-26)[2020-04-05].https://ec.europa.eu/digital-single-market/en/news/code-practice-disinformation.
[6] 国家互联网信息办公室等三部门规范网络音视频信息服务[J].中国广播,2019(12):39.
[7] HINTON G E,SALAKHUTDINOV R.Reducing the Dimen-sionality of Data with Neural Networks [J].Science,2006,313(5786):504-507.
[8] NGUYEN T T,NGUYEN C M,NGUYEN D T,et al.DeepLearning for Deepfakes Creation and Detection[J].arXiv:1909.11573,2019.
[9] GOODFELLOW I,POUGET-ABADIE J,MIRZA M,et al.Gen-erative Adversarial Nets[C]//Advances in Neural Information Processing Systems.2014:2672-2680.
[10] RADFORD A,METZ L,CHINTALA S.Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks[J].arXiv:1511.06434,2015.
[11] ARJOVSKY M,CHINTALA S,BOTTOU L.Wasserstein GAN[J].arXiv:1701.07875,2017.
[12] ZHU J Y,PARK T,ISOLA P,et al.Unpaired Image-to-image Translation Using Cycle-consistent Adversarial Networks[C]//Proceedings of the IEEE International Conference on Computer Vision.2017:2223-2232.
[13] WANG C,XU C,YAO X,et al.Evolutionary Generative Adversarial Networks[J].IEEE Transactions on Evolutionary Computation,2019,23(6):921-934.
[14] KARRAS T,LAINE S,AILA T.A Style-based Generator Architecture for Generative Adversarial Networks[C]//Procee-dings of the IEEE Conference on Computer Vision and Pattern Recognition.2019:4401-4410.
[15] Faceswap.Faceswap is the leading free and Open Source multi-platform Deepfakes software [EB/OL].(2019-08-15)[2020-04-23].https://faceswap.dev/.
[16] Shaoanlu.Faceswap-GAN[EB/OL].(2019-10-04)[2020-04-23].https://github.com/shaoanlu/faceswap-GAN.
[17] dfaker.DFaker[EB/OL].(2018-02-24)[2020-04-23].https://github.com/dfaker/df.
[18] SU M L.ZAO Privacy Protection Saves "big hole"[J].Computers & Networks,2019,45(17):8-10.
[19] ROSSLER A,COZZOLINO D,VERDOLIVA L,et al.Faceforensics++:Learning to Detect Manipulated Facial Images[C]//Proceedings of the IEEE International Conference on Computer Vision.2019:1-11.
[20] KORSHUNOV P,MARCEL S.Deepfakes:a New Threat toFace Recognition? Assessment and Detection[J].arXiv:1812.08685,2018.
[21] LI Y,YANG X,SUN P,et al.Celeb-df:A new Dataset for Deepfake Forensics[J].arXiv:1909.12962,2019.
[22] WANG X,DING H,CHEN H.A Shot Clustering Based App-roach for Scene Segmentation[J].Journal of Image and Gra-phics,2007,12(1):2127-2131.
[23] ZOLFAGHARI M,SINGH K,BROX T.Eco:Efficient Convolutional Network for Online Video Understanding[C]//Procee-dings of the European Confe-rence on Computer Vision (ECCV).2018:695-712.
[24] YU X D,WANG L,TIAN Q,et al.Multilevel Video Representation with Application to Keyframe Extraction[C]//Proceedings of the 10th International Multimedia Modelling Conference.IEEE,2004:117-123.
[25] YIN Y,JIANG H N.Key Frame Extraction Based on Clustering of Optimizing Initial Centers[J].Computer Engineering and Applications,2007(21):165-167.
[26] EJAZ N,BAIK S W,MAJEED H,et al.Multi-scale Contrastand Relative Motion-based Key Frame Extraction[J].EURASIP Journal on Image and Video Processing,2018,2018(1):40.
[27] WOLF W.Key Frame Selection by Motion Analysis[C]//Proceedings of the 1996 IEEE International Conference on Acoustics,Speech,and Signal Processing Conference.IEEE,1996:1228-1231.
[28] COOTES T F,TAYLOR C J,LANITIS A.Active Shape Models:Evaluation of a Multi-Resolution Method for Improving Image Search[C]//BMVC.1994:327-336.
[29] COOTES T F,EDWARDS G J,TAYLOR C J.Active Appearance Models[C]//European Conference on Computer Vision.Berlin:Springer,1998:484-498.
[30] KAZEMI V,SULLIVAN J.One Millisecond Face Alignmentwith an Ensemble of Regression Trees[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2014:1867-1874.
[31] CAO X,WEI Y,WEN F,et al.Face Alignment by ExplicitShape Regression[J].International Journal of Computer Vision,2014,107(2):177-190.
[32] BURGOS-ARTIZZU X P,PERONA P,DOLLÁR P.RobustFace Landmark Estimation under Occlusion[C]//Proceedings of the IEEE International Conference on Computer Vision.2013:1513-1520.
[33] SUN Y,WANG X,TANG X.Deep Convolutional Network Cascade for Facial Point Detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2013:3476-3483.
[34] ZHOU E,FAN H,CAO Z,et al.Extensive Facial Landmark Localization with Coarse-to-fine Convolutional Network Cascade[C]//Proceedings of the IEEE International Conference on Computer Vision Workshops.2013:386-391.
[35] MATERN F,RIESS C,STAMMINGER M.Exploiting Visual Artifacts to Expose Deepfakes and Face Manipulations[C]//Proceedings of 2019 IEEE Winter Applications of Computer Vision Workshops (WACVW).IEEE,2019:83-92.
[36] YANG X,LI Y,LYU S.Exposing Deepfakes Using Inconsistent Head Poses[C]//Proceedings of 2019 IEEE International Conference on Acoustics,Speech and Signal Processing (ICASSP).IEEE,2019:8261-8265.
[37] YANG X,LI Y,QI H,et al.Exposing Gan-synthesized Faces Using Landmark Locations[C]//Proceedings of the ACM Workshop on Information Hiding and Multimedia Security.2019:113-118.
[38] WADHWA N,WU H Y,DAVIS A,et al.Eulerian Video Magnification and Analysis[J].Communications of the ACM,2016,60(1):87-95.
[39] LI Y,LYU S.Exposing Deepfake Videos by Detecting FaceWarping Artifacts[J].arXiv:1811.00656,2018.
[40] SIMONYAN K,ZISSERMAN A.Very Deep Convolutional Networks for Large-scale Image Recognition[J].arXiv:1409.1556,2014.
[41] HE K,ZHANG X,REN S,et al.Deep Residual Learning forImage Recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:770-778.
[42] ZHOU P,HAN X,MORARIU V I,et al.Learning Rich Fea-tures for Image Manipulation Detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:1053-1061.
[43] AFCHAR D,NOZICK V,YAMAGISHI J,et al.Mesonet:ACompact Facial Video Forgery Detection Network[C]//2018 IEEE International Workshop on Information Forensics and Security (WIFS).IEEE,2018:1-7.
[44] ZHANG P,ZOU F,WU Z,et al.FeatherNets:ConvolutionalNeural Networks as Light as Feather for Face Anti-spoofing[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops.2019:1574-1583.
[45] BAYAR B,STAMM M C.Constrained Convolutional NeuralNetworks:A New Approach Towards General Purpose Image Manipulation Detection[J].IEEE Transactions on Information Forensics and Security,2018,13(11):2691-2706.
[46] NGUYEN H H,YAMAGISHI J,ECHIZEN I.Capsule-forensics:Using Capsule Networks to Detect Forged Images and Videos[C]//Proceedings of 2019 IEEE International Conference on Acoustics,Speech and Signal Processing (ICASSP).IEEE,2019:2307-2311.
[47] GÜERA D,DELP J.Deepfake Video Detection Using Recurrent Neural Networks[C]//2018 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).IEEE,2018:1-6.
[48] HOCHREITER S,SCHMIDHUBER J.Long Short-term Memory[J].Neural Computation,1997,9(8):1735-1780.
[49] SABIR E,CHENG J,JAISWAL A,et al.Recurrent Convolutional Strategies for Face Manipulation Detection in videos[J].Interfaces (GUI),2019,3:1.
[50] LI Y,CHANG M C,LYU S.In Ictu Oculi:Exposing AI Created Fake Videos by Detecting Eye Blinking[C]//2018 IEEE International Workshop on Information Forensics and Security (WIFS).IEEE,2018:1-7.
[51] AMERINI I,GALTERI L,CALDELLI R,et al.Deepfake Video Detection through Optical Flow based CNN[C]//Proceedings of the IEEE International Conference on Computer Vision Workshops.2019:1205-1207.
[52] HASAN H R,SALAH K.Combating Deepfake Videos Using Blockchain and Smart Contracts[J].IEEE Access,2019,7:41596-41606.
[53] KOOPMAN M,RODRIGUEZ A M,GERADTS Z.Detection of Deepfake Video Manipulation[C]//The 20th Irish Machine Vision and Image Processing Conference (IMVIP).2018:133-136.
[54] LEE S,YOO C D.Video Fingerprinting Based on Centroids of Gradient Orientations[C]//Proceedings of 2006 IEEE International Conference on AcousticsSpeech and Signal Processing.IEEE,2006,2:1-4.
[55] TANG W,WO Y,HAN G.Geometrically Robust Video Hashing Based on ST-PCT for Video Copy Detection[J].Multimedia Tools and Applications,2019,78(15):21999-22022.
[56] LI J,ZHANG H,WAN W,et al.Two-class 3D-CNN Classifiers Combination for Video Copy Detection[J].Multimedia Tools and Applications,2018,3:1-13.
[57] MA C,GU Y,GONG C,et al.Unsupervised Video Hashing via Deep Neural Network[J].Neural Processing Letters,2018,47(3):877-890.
[58] CHEN P,LIANG T,DAI J,et al.Forged Facial Video Detection Based on Global Temporal and Local Spatial Feature[J].Journal of Cyber Security,2020,5(2):73-83.

相关文章 15

[1]	徐涌鑫, 赵俊峰, 王亚沙, 谢冰, 杨恺. 时序知识图谱表示学习 Temporal Knowledge Graph Representation Learning 计算机科学, 2022, 49(9): 162-171. https://doi.org/10.11896/jsjkx.220500204
[2]	饶志双, 贾真, 张凡, 李天瑞. 基于Key-Value关联记忆网络的知识图谱问答方法 Key-Value Relational Memory Networks for Question Answering over Knowledge Graph 计算机科学, 2022, 49(9): 202-207. https://doi.org/10.11896/jsjkx.220300277
[3]	汤凌韬, 王迪, 张鲁飞, 刘盛云. 基于安全多方计算和差分隐私的联邦学习方案 Federated Learning Scheme Based on Secure Multi-party Computation and Differential Privacy 计算机科学, 2022, 49(9): 297-305. https://doi.org/10.11896/jsjkx.210800108
[4]	孙奇, 吉根林, 张杰. 基于非局部注意力生成对抗网络的视频异常事件检测方法 Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection 计算机科学, 2022, 49(8): 172-177. https://doi.org/10.11896/jsjkx.210600061
[5]	王剑, 彭雨琦, 赵宇斐, 杨健. 基于深度学习的社交网络舆情信息抽取方法综述 Survey of Social Network Public Opinion Information Extraction Based on Deep Learning 计算机科学, 2022, 49(8): 279-293. https://doi.org/10.11896/jsjkx.220300099
[6]	郝志荣, 陈龙, 黄嘉成. 面向文本分类的类别区分式通用对抗攻击方法 Class Discriminative Universal Adversarial Attack for Text Classification 计算机科学, 2022, 49(8): 323-329. https://doi.org/10.11896/jsjkx.220200077
[7]	姜梦函, 李邵梅, 郑洪浩, 张建朋. 基于改进位置编码的谣言检测模型 Rumor Detection Model Based on Improved Position Embedding 计算机科学, 2022, 49(8): 330-335. https://doi.org/10.11896/jsjkx.210600046
[8]	张源, 康乐, 宫朝辉, 张志鸿. 基于Bi-LSTM的期货市场关联交易行为检测方法 Related Transaction Behavior Detection in Futures Market Based on Bi-LSTM 计算机科学, 2022, 49(7): 31-39. https://doi.org/10.11896/jsjkx.210400304
[9]	胡艳羽, 赵龙, 董祥军. 一种用于癌症分类的两阶段深度特征选择提取算法 Two-stage Deep Feature Selection Extraction Algorithm for Cancer Classification 计算机科学, 2022, 49(7): 73-78. https://doi.org/10.11896/jsjkx.210500092
[10]	曾志贤, 曹建军, 翁年凤, 蒋国权, 徐滨. 基于注意力机制的细粒度语义关联视频-文本跨模态实体分辨 Fine-grained Semantic Association Video-Text Cross-modal Entity Resolution Based on Attention Mechanism 计算机科学, 2022, 49(7): 106-112. https://doi.org/10.11896/jsjkx.210500224
[11]	程成, 降爱莲. 基于多路径特征提取的实时语义分割方法 Real-time Semantic Segmentation Method Based on Multi-path Feature Extraction 计算机科学, 2022, 49(7): 120-126. https://doi.org/10.11896/jsjkx.210500157
[12]	侯钰涛, 阿布都克力木·阿布力孜, 哈里旦木·阿布都克里木. 中文预训练模型研究进展 Advances in Chinese Pre-training Models 计算机科学, 2022, 49(7): 148-163. https://doi.org/10.11896/jsjkx.211200018
[13]	周慧, 施皓晨, 屠要峰, 黄圣君. 基于主动采样的深度鲁棒神经网络学习 Robust Deep Neural Network Learning Based on Active Sampling 计算机科学, 2022, 49(7): 164-169. https://doi.org/10.11896/jsjkx.210600044
[14]	苏丹宁, 曹桂涛, 王燕楠, 王宏, 任赫. 小样本雷达辐射源识别的深度学习方法综述 Survey of Deep Learning for Radar Emitter Identification Based on Small Sample 计算机科学, 2022, 49(7): 226-235. https://doi.org/10.11896/jsjkx.210600138
[15]	王君锋, 刘凡, 杨赛, 吕坦悦, 陈峙宇, 许峰. 基于多源迁移学习的大坝裂缝检测 Dam Crack Detection Based on Multi-source Transfer Learning 计算机科学, 2022, 49(6A): 319-324. https://doi.org/10.11896/jsjkx.210500124

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed