计算机科学

计算机科学 ›› 2025, Vol. 52 ›› Issue (7): 353-362.doi: 10.11896/jsjkx.240800079

• 信息安全 • 上一篇    下一篇

基于多分类数据集的人脸伪造算法识别模型

丁博文, 芦天亮, 彭舒凡, 耿浩琦, 杨刚   

  1. 中国人民公安大学信息网络安全学院 北京 100038
  • 收稿日期:2024-08-14 修回日期:2024-10-18 发布日期:2025-07-17
  • 通讯作者: 芦天亮(lutianliang@ppsuc.edu.cn)
  • 作者简介:(3289582172@qq.com)
  • 基金资助:
    中国人民公安大学网络空间安全执法技术双一流创新研究专项(2023SYL07)

Face Forgery Algorithm Recognition Model Based on Multi-classification Dataset

DING Bowen, LU Tianliang, PENG Shufan, GENG Haoqi, YANG Gang   

  1. College of Information Network Security, People's Public Security University of China, Beijing 100038, China
  • Received:2024-08-14 Revised:2024-10-18 Published:2025-07-17
  • About author:DING Bowen,born in 2001, postgra- duate,is a member of CCF(No.U7883G).Her main research interests include deep forgery detection and cyber security.
    LU Tianliang,born in 1985,Ph.D,professor,Ph.D supervisor.His main research interests include cyber security and artificial intelligence.
  • Supported by:
    Double First-Class Innovation Research Project for People's Public Security University of China(2023SYL07).

PDF (PC)

25

摘要： 目前,人脸检测方法主要集中在人脸真假检测,对伪造算法识别的研究较少,存在图像扰动鲁棒性较差、资源占用大等问题;同时,公开的人脸检测数据集存在更新慢、种类少等问题。为解决以上问题,设计了人脸伪造算法识别模型Indentifor- mer。该模型以视觉自注意力模型为主干,首先将位置编码融合块分解,再使用Khatri-Rao积改进的快速傅里叶变换对全局特征进行提取,同时采用并行卷积结构补充局部特征信息并利用多头注意力机制进行融合,以增强模型的建模能力。最后,通过基于正则化改进的多层感知机减少过拟合,实现人脸伪造算法的识别。此外,构建了虚假人脸多分类数据集,其包含扩散模型、大模型及融合技术等18种伪造方法,共计41万余张人脸图像,具有更好的数据多样性和真假混合性。实验结果表明,Indentifomer模型在不增加资源开销的情况下,在算法识别多分类和真假分辨二分类任务中AUC分别达到99.57%和99.73%,在鲁棒性实验中AUC平均仅下降4.62%,具有较高的识别能力和抗干扰能力。

关键词: 人脸伪造算法识别, 深度伪造, 视觉自注意力, 人脸数据集, 多分类

Abstract: At present,the face detection methods mainly focus on the detection of the authenticity of faces,and there are few studies on the recognition of forgery algorithms,accompanied by poor image disturbance robustness and large resource occupation.What's more,the public face detection datasets have problems such as slow update and few types.In order to solve the above problems,this paper proposes a face forgery algorithm recognition model—Indentiformer,which takes the visual self-attention model as the backbone.It decomposes the position coding fusion block,and then uses Fast Fourier Transformation improved by the Khatri-Rao product to extract the global features.At the same time,the parallel convolutional structure is used to supplement the local feature information and the multi-head attention mechanism is used for fusion to enhance the modeling ability of the model.Finally,the overfitting is reduced by the improved multilayer perceptron based on regularization to realize the recognition of face forgery algorithms.In addition,this paper also constructs a multi-classification dataset of fake faces,including 18 forgery methods such as diffusion model,large model and fusion technology,with a total of more than 410 000 face images,which has better data diversity in true and false mixing.Experimental results show that the Indentifomer model achieves 99.57% and 99.73% AUC in the algorithm recognition multi-classification and true-false discrimination binary classification tasks without increasing resource overhead.In robustness experiments,the AUC decreases by only 4.62% on average,which has high recognition ability and anti-interference ability.

Key words: Face forgery algorithm recognition, Deepfake, Visual self-attention, Face dataset, Multi-classification

中图分类号: 

  • TP391.4
[1]KINGMA D P,WELLING M.Auto-encoding Variational Bayes[J].arXiv:1312.6114,2013.
[2]GOODFELLOW I,POUGET-ABADIE J,MIRZA M,et al.Ge-nerative Adversarial Networks[J].Communications of the ACM,2020,63(11):139-144.
[3]HO J,JAIN A,ABBEEL P.Denoising Diffusion ProbabilisticModels[J].Advances in Neural Information Processing Systems,2020,33:6840-6851.
[4]ZHANG Y W,CAI M C,CHEN Y H,et al.Deepfake Detection Method Integrating Multiple Parameter Efficient Fine-tuning Techniques[J].Journal of Frontiers of Computer Science and Technology,2024,18(12):3335-3347.
[5]HE Y,GAN B,CHEN S,et al.Forgerynet:A Versatile Benchmark for Comprehensive Forgery Analysis[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2021:4360-4369.
[6]ROSSLER A,COZZOLINO D,VERDOLIVA L,et al.Facefo-rensics++:Learning to Detect Manipulated Facial Images[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2019:1-11.
[7]DANG H,LIUF,STEHOUWER J,et al.On the Detection of Digital Face Manipulation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern recognition.2020:5781-5790.
[8]WANG J,WU Z,OUYANG W,et al.M2tr:Multi-modalmulti-scale Transformers for Deepfake Detection[C]//Proceedings of the 2022 International Conference on Multimedia Retrieval.2022:615-623.
[9]ZHOU Y,FAN B,ATREY P K,et al.Exposing Deepfakes Using Dual-channel Network with Multi-axis Attention and Frequency Analysis[C]//Proceedings of the 2023 ACM Workshop on Information Hiding and Multimedia Security.2023:169-174.
[10]TIAN C,LUO Z,SHI G,et al.Frequency-aware AttentionalFeature Fusion for Deepfake Detection[C]//ICASSP 2023-2023 IEEE International Conference on Acoustics,Speech and Signal Processing(ICASSP).IEEE,2023:1-5.
[11]ZHAO Z J,FAN Z F,DING B,et al.Deepfake Detection Based on Incremental Learning[J].Frontiers of Data & Computing,2023,5(6):42-57.
[12]HENDRYCKS D,GIMPEL K.GaussianError Linear Units(GELUs)[J].aXiv:1606.08415,2016.
[13]KARRAS T,LAINE S,AILA T.AStyle-based Generator Ar-chitecture for Generative Adversarial Networks[C]// Procee- dings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2019:4401-4410.
[14]LIU Z,LUO P,WANG X,et al.DeepLearning Face Attributes in the Wild[C]//Proceedings of the IEEE International Confe- rence on Computer Vision.2015:3730-3738.
[15]HE Z,ZUO W,KAN M,et al.Attgan:Facial Attribute Editing by Only Changing What You Want[J].IEEE Transactions on Image Processing,2019,28(11):5464-5478.
[16]ANOKHIN I,DEMOCHKIN K,KHAKHULIN T,et al.Image Generators with Conditionally-independent Pixel Synthesis[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2021:14278-14287.
[17]XIAO Z,KREIS K,VAHDAT A.Tackling the GenerativeLearning Trilemma with Denoising Diffusion Gans[J].arXiv:2112.07804,2021.
[18]KARRAS T,AITTALA M,HELLSTEN J,et al.Training Ge-nerative Adversarial Networks with Limited Data[J].Advances in Neural Information Processing Systems,2020,33:12104-12114.
[19]WOOD E,BALTRUŠAITIS T,HEWITT C,et al.Fake It Till You Make It:Face Analysis in the Wild Using Synthetic Data Alone[C]//Proceedings of the IEEE/CVF International Confe- rence on Computer Vision.2021:3681-3691.
[20]HUDSON D A,ZITNICK L.Generative Adversarial Trans-formers[C]//International Conference on Machine Learning.PMLR,2021:4487-4499.
[21]SUVOROV R,LOGACHEVA E,MASHIKHIN A,et al.Resolution-robust Large Mask Inpainting with Fourier Convolutions[C]//Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision.2022:2149-2159.
[22]LIU D B.ChatGLM[EB/OL].https://chatglm.cn/main/alltoolsdetail?lang=zh.
[23]SAHARIA C,CHAN W,CHANG H,et al.Palette:Image-to-image Diffusion Models[C]//ACM SIGGRAPH 2022 Conference Proceedings.2022:1-10.
[24]SAUER A,CHITTA K,MüLLER J,et al.Projected Gans Converge Faster[J].Advances in Neural Information Processing Systems,2021,34:17480-17492.
[25]BENIAGUEV D.Synthetic Faces High Quality(sfhq) Dataset[DB/OL].https://github.com/SelfishGene/SFHQ-dataset.
[26]ROMBACH R,BlATTMANN A,LORENZ D,et al.High-resolution Image Synthesis with Latent Diffusion Models[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2022:10684-10695.
[27]CHOI Y,CHOI M,KIM M,et al.Stargan:Unified GenerativeAdversarial Networks for Multi-Domain Image-to-image Translation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:8789-8797.
[28]WANG T Y,CHENG H,CHOW K P,et al.Deep Convolutional Pooling Transformer for Deepfake Detection[J].ACM Transactions on Multimedia Computing,Communications,and Applications,2023,19(6):1-20.
[29]YANG X,LI Y,LYU S.ExposingDeep Fakes Using Inconsistent Head Poses[C]//2019 IEEE International Conference on Acoustics,Speech and Signal Processing(ICASSP 2019).IEEE,2019:8261-8265.
[30]KORSHUNOV P,MARCEL S.Deepfakes:A New Threat to Face Recognition? Assessment and Detection[J].arXiv:1812.08685,2018.
[31]ZHOU P,HAN X,MORARIU V I,et al.Two-Stream Neural Networks for Tampered Face Detection[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops(CVPRW).IEEE,2017:1831-1839.
[32]WANG R,MA L,JUEFEI-XU F,et al.Fakespotter:A Simple Baseline for Spotting Ai-synthesized Fake Faces[J].arXiv:1909.06122,2019.
[33]LI Y,YANG X,SUN P,et al.Celeb-DF:A Large-scale Chal-lenging Dataset for Deepfake Forensics[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:3207-3216.
[34]DOLHANSKY B,BITTON J,PFLAUM B,et al.TheDeepfake Detection Challenge(dfdc) Dataset[J].arXiv:2006.07397,2020.
[35]VAN DER MAATEN L,HINTON G.Visualizing Data Using T-SNE[J].Journal of Machine Learning Research,2008,9(86):2579-2605.
[36]KARRAS T,LAINE S,AILA T.A Style-based Generator Architecture for Generative Adversarial Networks[C]// Procee- dings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2019:4401-4410.
[37]QIANY Y,YIN G J,SHENG L,et al.Thinking in Frequency:Face Forgery Detection by Mining Frequency-aware Clues[C]//European Conference on Computer Vision.Cham:Springer,2020:86-103.
[38]LI X R,JI S L,WU C M,et al.Survey on Deepfakes and Detection Techniques[J].Journal of Software,2021,32(2):496-518.
[39]XING H,LI M,Deepfake Video Detection based on 3D Convolutional Neural Networks[J].Computer Science,2021,48(7):86-92.
[40]COCCOMINI D A,MESSINA N,GENNARO C,et al.Combining EfficientNet and Vision Transformers for Video Deepfake Detection[C]//Proceedings of the 21st International Conference on Image Analysis and Processing.2022:219-229.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市渝北区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发