Computer Science ›› 2025, Vol. 52 ›› Issue (7): 353-362.doi: 10.11896/jsjkx.240800079

• Information Security • Previous Articles     Next Articles

Face Forgery Algorithm Recognition Model Based on Multi-classification Dataset

DING Bowen, LU Tianliang, PENG Shufan, GENG Haoqi, YANG Gang   

  1. College of Information Network Security, People's Public Security University of China, Beijing 100038, China
  • Received:2024-08-14 Revised:2024-10-18 Published:2025-07-17
  • About author:DING Bowen,born in 2001, postgra- duate,is a member of CCF(No.U7883G).Her main research interests include deep forgery detection and cyber security.
    LU Tianliang,born in 1985,Ph.D,professor,Ph.D supervisor.His main research interests include cyber security and artificial intelligence.
  • Supported by:
    Double First-Class Innovation Research Project for People's Public Security University of China(2023SYL07).

Abstract: At present,the face detection methods mainly focus on the detection of the authenticity of faces,and there are few studies on the recognition of forgery algorithms,accompanied by poor image disturbance robustness and large resource occupation.What's more,the public face detection datasets have problems such as slow update and few types.In order to solve the above problems,this paper proposes a face forgery algorithm recognition model—Indentiformer,which takes the visual self-attention model as the backbone.It decomposes the position coding fusion block,and then uses Fast Fourier Transformation improved by the Khatri-Rao product to extract the global features.At the same time,the parallel convolutional structure is used to supplement the local feature information and the multi-head attention mechanism is used for fusion to enhance the modeling ability of the model.Finally,the overfitting is reduced by the improved multilayer perceptron based on regularization to realize the recognition of face forgery algorithms.In addition,this paper also constructs a multi-classification dataset of fake faces,including 18 forgery methods such as diffusion model,large model and fusion technology,with a total of more than 410 000 face images,which has better data diversity in true and false mixing.Experimental results show that the Indentifomer model achieves 99.57% and 99.73% AUC in the algorithm recognition multi-classification and true-false discrimination binary classification tasks without increasing resource overhead.In robustness experiments,the AUC decreases by only 4.62% on average,which has high recognition ability and anti-interference ability.

Key words: Face forgery algorithm recognition, Deepfake, Visual self-attention, Face dataset, Multi-classification

CLC Number: 

  • TP391.4
[1]KINGMA D P,WELLING M.Auto-encoding Variational Bayes[J].arXiv:1312.6114,2013.
[2]GOODFELLOW I,POUGET-ABADIE J,MIRZA M,et al.Ge-nerative Adversarial Networks[J].Communications of the ACM,2020,63(11):139-144.
[3]HO J,JAIN A,ABBEEL P.Denoising Diffusion ProbabilisticModels[J].Advances in Neural Information Processing Systems,2020,33:6840-6851.
[4]ZHANG Y W,CAI M C,CHEN Y H,et al.Deepfake Detection Method Integrating Multiple Parameter Efficient Fine-tuning Techniques[J].Journal of Frontiers of Computer Science and Technology,2024,18(12):3335-3347.
[5]HE Y,GAN B,CHEN S,et al.Forgerynet:A Versatile Benchmark for Comprehensive Forgery Analysis[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2021:4360-4369.
[6]ROSSLER A,COZZOLINO D,VERDOLIVA L,et al.Facefo-rensics++:Learning to Detect Manipulated Facial Images[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2019:1-11.
[7]DANG H,LIUF,STEHOUWER J,et al.On the Detection of Digital Face Manipulation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern recognition.2020:5781-5790.
[8]WANG J,WU Z,OUYANG W,et al.M2tr:Multi-modalmulti-scale Transformers for Deepfake Detection[C]//Proceedings of the 2022 International Conference on Multimedia Retrieval.2022:615-623.
[9]ZHOU Y,FAN B,ATREY P K,et al.Exposing Deepfakes Using Dual-channel Network with Multi-axis Attention and Frequency Analysis[C]//Proceedings of the 2023 ACM Workshop on Information Hiding and Multimedia Security.2023:169-174.
[10]TIAN C,LUO Z,SHI G,et al.Frequency-aware AttentionalFeature Fusion for Deepfake Detection[C]//ICASSP 2023-2023 IEEE International Conference on Acoustics,Speech and Signal Processing(ICASSP).IEEE,2023:1-5.
[11]ZHAO Z J,FAN Z F,DING B,et al.Deepfake Detection Based on Incremental Learning[J].Frontiers of Data & Computing,2023,5(6):42-57.
[12]HENDRYCKS D,GIMPEL K.GaussianError Linear Units(GELUs)[J].aXiv:1606.08415,2016.
[13]KARRAS T,LAINE S,AILA T.AStyle-based Generator Ar-chitecture for Generative Adversarial Networks[C]// Procee- dings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2019:4401-4410.
[14]LIU Z,LUO P,WANG X,et al.DeepLearning Face Attributes in the Wild[C]//Proceedings of the IEEE International Confe- rence on Computer Vision.2015:3730-3738.
[15]HE Z,ZUO W,KAN M,et al.Attgan:Facial Attribute Editing by Only Changing What You Want[J].IEEE Transactions on Image Processing,2019,28(11):5464-5478.
[16]ANOKHIN I,DEMOCHKIN K,KHAKHULIN T,et al.Image Generators with Conditionally-independent Pixel Synthesis[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2021:14278-14287.
[17]XIAO Z,KREIS K,VAHDAT A.Tackling the GenerativeLearning Trilemma with Denoising Diffusion Gans[J].arXiv:2112.07804,2021.
[18]KARRAS T,AITTALA M,HELLSTEN J,et al.Training Ge-nerative Adversarial Networks with Limited Data[J].Advances in Neural Information Processing Systems,2020,33:12104-12114.
[19]WOOD E,BALTRUŠAITIS T,HEWITT C,et al.Fake It Till You Make It:Face Analysis in the Wild Using Synthetic Data Alone[C]//Proceedings of the IEEE/CVF International Confe- rence on Computer Vision.2021:3681-3691.
[20]HUDSON D A,ZITNICK L.Generative Adversarial Trans-formers[C]//International Conference on Machine Learning.PMLR,2021:4487-4499.
[21]SUVOROV R,LOGACHEVA E,MASHIKHIN A,et al.Resolution-robust Large Mask Inpainting with Fourier Convolutions[C]//Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision.2022:2149-2159.
[22]LIU D B.ChatGLM[EB/OL].https://chatglm.cn/main/alltoolsdetail?lang=zh.
[23]SAHARIA C,CHAN W,CHANG H,et al.Palette:Image-to-image Diffusion Models[C]//ACM SIGGRAPH 2022 Conference Proceedings.2022:1-10.
[24]SAUER A,CHITTA K,MüLLER J,et al.Projected Gans Converge Faster[J].Advances in Neural Information Processing Systems,2021,34:17480-17492.
[25]BENIAGUEV D.Synthetic Faces High Quality(sfhq) Dataset[DB/OL].https://github.com/SelfishGene/SFHQ-dataset.
[26]ROMBACH R,BlATTMANN A,LORENZ D,et al.High-resolution Image Synthesis with Latent Diffusion Models[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2022:10684-10695.
[27]CHOI Y,CHOI M,KIM M,et al.Stargan:Unified GenerativeAdversarial Networks for Multi-Domain Image-to-image Translation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:8789-8797.
[28]WANG T Y,CHENG H,CHOW K P,et al.Deep Convolutional Pooling Transformer for Deepfake Detection[J].ACM Transactions on Multimedia Computing,Communications,and Applications,2023,19(6):1-20.
[29]YANG X,LI Y,LYU S.ExposingDeep Fakes Using Inconsistent Head Poses[C]//2019 IEEE International Conference on Acoustics,Speech and Signal Processing(ICASSP 2019).IEEE,2019:8261-8265.
[30]KORSHUNOV P,MARCEL S.Deepfakes:A New Threat to Face Recognition? Assessment and Detection[J].arXiv:1812.08685,2018.
[31]ZHOU P,HAN X,MORARIU V I,et al.Two-Stream Neural Networks for Tampered Face Detection[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops(CVPRW).IEEE,2017:1831-1839.
[32]WANG R,MA L,JUEFEI-XU F,et al.Fakespotter:A Simple Baseline for Spotting Ai-synthesized Fake Faces[J].arXiv:1909.06122,2019.
[33]LI Y,YANG X,SUN P,et al.Celeb-DF:A Large-scale Chal-lenging Dataset for Deepfake Forensics[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:3207-3216.
[34]DOLHANSKY B,BITTON J,PFLAUM B,et al.TheDeepfake Detection Challenge(dfdc) Dataset[J].arXiv:2006.07397,2020.
[35]VAN DER MAATEN L,HINTON G.Visualizing Data Using T-SNE[J].Journal of Machine Learning Research,2008,9(86):2579-2605.
[36]KARRAS T,LAINE S,AILA T.A Style-based Generator Architecture for Generative Adversarial Networks[C]// Procee- dings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2019:4401-4410.
[37]QIANY Y,YIN G J,SHENG L,et al.Thinking in Frequency:Face Forgery Detection by Mining Frequency-aware Clues[C]//European Conference on Computer Vision.Cham:Springer,2020:86-103.
[38]LI X R,JI S L,WU C M,et al.Survey on Deepfakes and Detection Techniques[J].Journal of Software,2021,32(2):496-518.
[39]XING H,LI M,Deepfake Video Detection based on 3D Convolutional Neural Networks[J].Computer Science,2021,48(7):86-92.
[40]COCCOMINI D A,MESSINA N,GENNARO C,et al.Combining EfficientNet and Vision Transformers for Video Deepfake Detection[C]//Proceedings of the 21st International Conference on Image Analysis and Processing.2022:219-229.
[1] YU Yang, YUAN Jiabin, CAI Jiyuan, ZHA Keke, CHEN Zhangyu, DAI Jiawei, FENG Yuxiang. Deepfake Face Tampering Video Detection Method Based on Non-critical Masks and AttentionMechanism [J]. Computer Science, 2023, 50(11): 160-167.
[2] LIANG Haowei, WANG Shi, CAO Cungen. Study on Short Text Classification with Imperfect Labels [J]. Computer Science, 2023, 50(1): 185-193.
[3] SHI Da, LU Tian-liang, DU Yan-hui, ZHANG Jian-ling, BAO Yu-xuan. Generation Model of Gender-forged Face Image Based on Improved CycleGAN [J]. Computer Science, 2022, 49(2): 31-39.
[4] LI Ying, BIAN Shan, WANG Chun-tao, HUANG Qiong. Detection of Deepfakes Based on Dual-stream Network [J]. Computer Science, 2022, 49(11A): 220100106-9.
[5] BAO Yu-xuan, LU Tian-liang, DU Yan-hui, SHI Da. Deepfake Videos Detection Method Based on i_ResNet34 Model and Data Augmentation [J]. Computer Science, 2021, 48(7): 77-85.
[6] XING Hao, LI Ming. Deepfake Video Detection Based on 3D Convolutional Neural Networks [J]. Computer Science, 2021, 48(7): 86-92.
[7] BAO Yu-xuan, LU Tian-liang, DU Yan-hui. Overview of Deepfake Video Detection Technology [J]. Computer Science, 2020, 47(9): 283-292.
[8] LI Bao-sheng, QIN Chuan-dong. Study on Electric Vehicle Price Prediction Based on PSO-SVM Multi-classification Method [J]. Computer Science, 2020, 47(11A): 421-424.
[9] CHEN Li-fang, CHEN Liang and LIU Bao-xiang. Research of SVM Multiclass Model Based on Granular Computing & Huffman Tree [J]. Computer Science, 2016, 43(1): 64-68.
[10] . Study on Acoustic Sources Identification Based on Multi-SVDD [J]. Computer Science, 2012, 39(11): 233-236.
[11] YIN Zhi-Wu ,HUANG Shang-Teng (Department of Computer Science and Engineering, Shanghai Jiaotong University, Shanghai 200240). [J]. Computer Science, 2008, 35(2): 138-139.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!