计算机科学

计算机科学 ›› 2023, Vol. 50 ›› Issue (6): 216-224.doi: 10.11896/jsjkx.220400268

• 计算机图形学&多媒体 • 上一篇    下一篇

基于空频联合卷积神经网络的GAN生成人脸检测

王金伟1,2,3, 曾可慧1, 张家伟1, 罗向阳3, 马宾4   

  1. 1 南京信息工程大学计算机学院、软件学院、网络空间安全学院 南京 210044
    2 南京信息工程大学江苏省大气环境与装备技术协同创新中心 南京 210044
    3 数学工程与高级计算国家重点实验室 郑州 450001
    4 齐鲁工业大学山东省计算机网络重点实验室 济南 250353
  • 收稿日期:2022-04-26 修回日期:2022-10-09 出版日期:2023-06-15 发布日期:2023-06-06
  • 通讯作者: 罗向阳(xiangyangluo@126.com)
  • 作者简介:(wjwei_2004@163.com)
  • 基金资助:
    国家自然科学基金(62072250,62172435,U1804263,U20B2065,61872203,71802110,61802212);中原科技创新领军人才项目(214200510019);江苏省自然科学基金(BK20200750);河南省网络空间态势感知重点实验室开放基金(HNTS2022002);江苏省研究生研究与实践创新项目(KYCX200974));广东省信息安全技术重点实验室开放项目(2020B1212060078);人文社会科学教育部项目(19YJA630061);江苏高校优势学科建设工程项目

GAN-generated Face Detection Based on Space-Frequency Convolutional Neural Network

WANG Jinwei1,2,3, ZENG Kehui1, ZHANG Jiawei1, LUO Xiangyang3, MA Bin4   

  1. 1 College of Computer,College of Software,College of Cyberspace Security,Nanjing University of Information Science,Technology,Nanjing 210044,China
    2 Jiangsu Collaborative Innovation Center of Atmospheric Environment,Equipment Technology,Nanjing University of Information Science,Technology,Nanjing 210044,China
    3 State Key Laboratory of Mathematical Engineering and Advanced Computing,Zhenzhou 450001,China
    4 Shandong Provincial Key Laboratory of Computer Networks,Qilu University of Technology,Jinan 250353,China
  • Received:2022-04-26 Revised:2022-10-09 Online:2023-06-15 Published:2023-06-06
  • About author:WANG Jinwei,born in 1978,Ph.D,professor,Ph.D supervisor,is a member of China Computer Federation.His main research interests include artificial intelligence security,color image forensics,color image reversible watermark,robust watermark and image encryption.LUO Xiangyang,born in 1978,Ph.D,professor,Ph.D supervisor,is a member of China Computer Federation.His main research interests include network and information security.
  • Supported by:
    National Natural Science Foundation of China(62072250,62172435,U1804263,U20B2065,61872203,71802110,61802212),Zhongyuan Science and Technology Innovation Leading Talent Project of China(214200510019),Natural Science Foundation of Jiangsu Province,China(BK20200750),Open Foundation of Henan Key Laboratory of Cyberspace Situation Awareness(HNTS2022002),Post Graduate Research & Practice Innvoation Program of Jiangsu Province(KYCX200974),Opening Project of Guangdong Province Key Laboratory of Information Security Technology(2020B1212060078),Ministry of Education of Humanities and Social Science Project(19YJA630061) and Priority Academic Program Development of Jiangsu Higher Education Institutions(PAPD).

PDF (PC)

310

摘要： 生成式对抗网络(GAN)的快速发展使其在图像生成领域取得了前所未有的成功。StyleGAN等新型GAN的出现使得生成的图像更真实且具有欺骗性,对国家安全、社会稳定和个人隐私都构成了较大威胁。文中提出了一种基于空频联合的双流卷积神经网络的检测模型。鉴于GAN图像在生成过程中因上采样操作在频谱上留下了清晰可辨的伪影,设计了可学习的频率域滤波核以及频率域网络来充分学习并提取频率域特征。为了减弱图像变换至频域过程中丢弃部分信息而带来的影响,同样设计了空间域网络来学习图像内容本身具有差异化的空间域特征,最终将两种特征融合来实现对GAN生成人脸图像的检测。在多个数据集上的实验结果表明,所提模型在高质量生成数据集上的检测精度及在跨数据集的泛化性上都优于现有算法,且对于JPEG压缩、随机剪裁、高斯模糊等图像变换具有更强的鲁棒性。不仅如此,所提方案在GAN生成的局部人脸数据集上也有不错表现,进一步证明了所提模型有着更好的通用性以及更加广泛的应用前景。

关键词: 数字图像取证, 人脸伪造检测, 卷积神经网络, 生成式对抗网络, 频率域

Abstract: The rapid development of generative adversarial networks(GANs) has led to unprecedented success in the field of image generation.The emergence of new GANs such as StyleGAN makes the generated images more realistic and deceptive,posing a greater threat to national security,social stability,and personal privacy.In this paper,a detection algorithm based on a space-frequency joint two-stream convolutional neural network is proposed.Since GAN images will leave clearly discernible artifacts on the spectrum due to the up-sampling operation during the generation process,a learnable frequency-domain filter kernel and frequency domain network are designed to fully learn and extract frequency-domain features.In order to reduce the influence of the information discarded from the image transformation to the frequency domain,a spatial domain network is also designed to learn that the image content itself has differentiated spatial domain features.Finally,the two features are fused to detect the face image generated by GAN.Experimental results on multiple datasets show that the proposed model outperforms existing algorithms in detection accuracy on high-quality generated datasets and generalization across datasets.And for JPEG compression,random cropping,Gaussian blur,and other operations,this method has stronger robustness.In addition,the proposed method also performs well on the local face dataset generated by GAN,which further proves that this model has better generality and wider application prospects.

Key words: Digital image forensics, Face forgery detection, Convolutional neural network, Generative adversarial networks, Frequency domain

中图分类号: 

  • TP391.41
[1]STUPP C.Fraudsters used AI to mimic CEO's voice in unusual cybercrime case[J].The Wall Street Journal,2019,30(8).
[2]KARRAS T,AILA T,LAINE S,et al.Progressive growing of gans for improved quality,stability,and variation[J].arXiv:1710.10196,2017.
[3]MARRA F,GRAGNANIELLO D,COZZOLINO D,et al.Detection of GAN-generated fake images over social networks[C]//2018 IEEE Conference on Multimedia Information Processing and Retrieval(MIPR).IEEE,2018:384-389.
[4]MCCLOSKEY S,ALBRIGHT M.Detecting GAN-generatedimagery using saturation cues[C]//2019 IEEE International Conference on Image Processing(ICIP).IEEE,2019:4584-4588.
[5]BARNI M,KALLAS K,NOWROOZI E,et al.CNN detection of GAN-generated face images based on cross-band co-occurrences analysis[C]//2020 IEEE International Workshop on Information Forensics and Security(WIFS).IEEE,2020:1-6.
[6]GUO Z,YANG G,CHEN J,et al.Fake face detection via adaptive manipulation traces extraction network[J].Computer Vision and Image Understanding,2021,204:103170.
[7]FRANK J,EISENHOFER T,SCHÖNHERR L,et al.Levera-ging frequency analysis for deep fake image recognition[C]//International Conference on Machine Learning.PMLR,2020:3247-3258.
[8]LI H,LI B,TAN S,et al.Detection of deep network generated images using disparities in color components[J].arXiv:1808.07276.
[9]MO H,CHEN B,LUO W.Fake faces identification via convolutional neural network[C]//Proceedings of the 6th ACM Workshop on Information Hiding and Multimedia Security.2018:43-47.
[10]NATARAJ L,MOHAMMED T M,MANJUNATH B S,et al.Detecting GAN generated fake images using co-occurrence matrices[J].Electronic Imaging,2019(5):532-1-532-7.
[11]FU Y,SUN T,JIANG X,et al.Robust gan-face detection based on dual-channel cnn network[C]//2019 12th International Congress on Image and Signal Processing,BioMedical Engineering and Informatics(CISP-BMEI).IEEE,2019:1-5.
[12]WANG S Y,WANG O,ZHANG R,et al.Cnn-generated images are surprisingly easy to spot…for now[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:8695-8704.
[13]WANG S Y,WANG O,ZHANG R,et al.Cnn-generated images are surprisingly easy to spot for now[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2021:8695-8704.
[14]AGARWAL S,GIRDHAR N,RAGHAV H.A Novel Neural Model based Framework for Detection of GAN Generated Fake Images[C]//2021 11th International Conference on Cloud Computing,Data Science & Engineering(Confluence).IEEE,2021:46-51.
[15]BONETTINI N,BESTAGINI P,MILANI S,et al.On the use of Benford's law to detect GAN-generated images[C]//2020 25th International Conference on Pattern Recognition(ICPR).IEEE,2020:5495-5502.
[16]DURALL R,KEUPER M,KEUPER J.Watch your up-convolution:Cnn based generative deep neural networks are failing to reproduce spectral distributions[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:7890-7899.
[17]HE P S,LI W C,ZHANG J Y,et al.Overview of passive forensics and anti-forensics techniques for GAN-generated image[J].Journal of Image and Graphics,2022,27(1):0088-0110.
[18]RADFORD A,METZ L,CHINTALA S.Unsupervised representation learning with deep convolutional generative adversarial networks[J].arXiv:1511.06434,2015.
[19]ODENA A,DUMOULIN V,OLAH C.Deconvolution andcheckerboard artifacts[J].Distill,2016,1(10):e3.
[20]ZHANG X,KARAMAN S,CHANG S F.Detecting and simulating artifacts in gan fake images[C]//2019 IEEE International Workshop on Information Forensics and Security(WIFS).IEEE,2019:1-6.
[21]LI C L,RAVANBAKHSH S,POCZOS B.Annealing Gaussian into ReLU:a new sampling strategy for leaky-ReLU RBM[J].arXiv:1611.03879,2016.
[22]KARRAS T,LAINE S,AITTALA M,et al.Analyzing and improving the image quality of stylegan[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:8110-8119.
[23]KARRAS T,LAINE S,AILA T.A style-based generator architecture for generative adversarial networks[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2019:4401-4410.
[24]ARJOVSKY M,CHINTALA S,BOTTOU L.Wasserstein ge-nerative adversarial networks[C]//International Conference on
Machine Learning.PMLR,2017:214-223.
[25]BERTHELOT D,SCHUMM T,METZ L.Began:Boundaryequilibrium generative adversarial networks[J].arXiv:1703.10717,2017.
[26]MAO X,LI Q,XIE H,et al.Least squares generative adversarial networks[C]//Proceedings of the IEEE International Confe-rence on Computer Vision.2017:2794-2802.
[27]SUN Y,CHEN Y,WANG X,et al.Deeplearning face representation by joint identification-verification[J/OL].Advances in Neural Information Processing Systems,2014,27.https://proceedings.neurips.cc/paper/2014/hash/e5e63da79fcd2bebbd7cb8bf1c1d0274-Abstract.html.
[28]CHEN B,JU X,XIAO B,et al.Locally GAN-generated face detection based on an improved Xception[J].Information Sciences,2021,572:16-28.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市渝北区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发