计算机科学

计算机科学 ›› 2025, Vol. 52 ›› Issue (6): 405-413.doi: 10.11896/jsjkx.241200001

• 信息安全 • 上一篇    

基于妆容风格补丁激活的对抗性人脸隐私保护

袁霖1, 黄令1, 郝凯乐1, 张家伟1, 朱明瑞2, 王楠楠1,2, 高新波1   

  1. 1 重庆邮电大学图像认知重庆市重点实验室 重庆 400065
    2 西安电子科技大学空天地一体化综合业务网全国重点实验室 西安 710071
  • 收稿日期:2024-12-02 修回日期:2025-02-17 出版日期:2025-06-15 发布日期:2025-06-11
  • 通讯作者: 高新波(gaoxb@cqupt.edu.cn)
  • 作者简介:(yuanlin@cqupt.edu.cn)
  • 基金资助:
    国家自然科学基金(62201107,U22A2096);重庆市自然科学基金面上项目(CSTB2022NSCQ-MSX1265);重庆市教育委员会科学技术研究项目(KJQN202300606)

Adversarial Face Privacy Protection Based on Makeup Style Patch Activation

YUAN Lin1, HUANG Ling1, HAO Kaile1, ZHANG Jiawei1, ZHU Mingrui2, WANG Nannan1,2, GAO Xinbo1   

  1. 1 Chongqing Key Laboratory of Image Cognition,Chongqing University of Posts and Telecommunications,Chongqing 400065,China
    2 State Key Laboratory of Integrated Services Networks,Xidian University,Xi'an 710071,China
  • Received:2024-12-02 Revised:2025-02-17 Online:2025-06-15 Published:2025-06-11
  • About author:YUAN Lin,born in 1989,Ph.D,asso-ciate professor.His main research in-terests include image and video proce-ssing,computer vision,multimedia security and privacy protection.
    GAO Xinbo,born in 1972,Ph.D,professor,Ph.D supervisor.His main research interests include artificial intelligence,machine learning,computer vision and pattern recognition.
  • Supported by:
    National Natural Science Foundation of China(62201107,U22A2096),Natural Science Foundation of Chongqing(CSTB2022NSCQ-MSX1265) and Science and Technology Research Program of Chongqing Municipal Education Commission(KJQN202300606).

PDF (PC)

116

摘要： 人脸识别技术的飞速发展极大地便利了人们的生活,但也引发了大众对个人隐私的担忧。人们通过社交媒体和网络发布的人脸图像可能会遭到不法机构的收集,并被人脸识别系统识别出身份从而窃取与用户相关的隐私信息。因此,需要一种隐私保护机制,使得用户通过公开媒体发布的人脸图像能够被正常观看,却可以防止人脸识别系统从中提取准确的身份信息。主流的基于对抗样本的方法在某种程度上能够解决这一问题,但难免会在图像中引入可被轻易察觉的噪声。人们通过社交媒体等平台分享个人照片时往往会加入一些美颜特效,因此,在为图像添加美化效果的同时巧妙地嵌入对抗性扰动,从而实现对图片的身份隐私保护是一种两全的选择。对此,提出了一种基于妆容风格补丁激活的人脸图像身份隐私保护方法。该方法将参考人脸图像的妆容风格,通过特征补丁的方式激活到原始人脸图像的特征中,再将激活后的特征重建为含妆容的对抗性人脸图像,同时利用身份隐私增强模块,通过迫使生成图像的身份特征逼近一个目标身份从而获得对抗性隐私保护能力。实验结果表明,该方法生成的人脸图像不仅具有更好的视觉效果和多样化的妆容风格,还能够有效防御多种黑盒人脸识别模型造成的隐私侵犯。

关键词: 面部隐私, 妆容风格, 特征补丁, 身份隐私保护, 黑盒人脸识别模型

Abstract: Facial recognition technology has developed rapidly,greatly facilitating people's lives,but it has also raised public concerns about personal privacy.Facial images shared by people through social media and the Internet may be collected by illegal organizations,which can use facial recognition systems to identify the identityand steal privacy information related to the users.Therefore,a privacy protection mechanism is needed to ensure that facial images published by users through public media can be viewed normally by people,but can prevent facial recognition systems from extracting accurate identity information.The mainstream adversarial sample-based methods can solve this problem to some extents,but they inevitably introduce noise that can be easily detected in the images.When people share personal photos on social media and other platforms,they often add some beauty effects.Therefore,embedding adversarial perturbations cleverly while adding beautification effects to the images to achieve identity privacy protection for the images is a win-win choice.In this regard,this paper proposes a facial image identity privacy protection method based on makeup style patch activation.This method activates the makeup style of the reference facial image into the features of the original facial image through feature patches,and then reconstructs the activated features into adversarial facial images with makeup.At the same time,it uses an identity privacy enhancement module to force the generated image's identity features to approach a target identity,thereby obtaining adversarial privacy protection capabilities.Experimental results show that the facial images generated by this method not only have better visual effects and a variety of makeup styles,but also can effectively defend against privacy infringement caused by various black-box facial recognition models.

Key words: Facial privacy, Makeup style, Feature patch, Identity privacy protection, Black-box face recognition model

中图分类号: 

  • TP751.1
[1]ZHANG S,FENG Y,BAUER L,et al.“Did you know this ca-mera tracks your mood?”:Understanding Privacy Expectations and Preferences in the Age of Video Analytics[C]//Proceedings on Privacy Enhancing Technologies.2021.
[2]WU Z,WANG Z,WANG Z,et al.Towards Privacy-Preserving Visual Recognition via Adversarial Training:A Pilot Study[C]//European Conference on Computer Vision.Cham:Springer,2018:627-645.
[3]MEDEN B,ROT P,TERHÖRST P,et al.Privacy-EnhancingFace Biometrics:A Comprehensive Survey[J].IEEE Transactions on Information Forensics and Security,2021,16:4147-4183.
[4]AGRAWAL P,NARAYANAN P J.Person De-Identification in Videos[J].IEEE Transactions on Circuits and Systems for Vi-deo Technology,2011,21(3):299-310.
[5]YUAN L,KORSHUNOV P,EBRAHIMI T.Secure JPEGscrambling enabling privacy in photo sharing[C]//2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition.2015:1-6.
[6]YUAN L,EBRAHIMI T.Image transmorphing with JPEG[C]//2015 IEEE International Conference on Image Processing(ICIP).2015:3956-3960.
[7]YUAN L,EBRAHIMI T.Image privacy protection with secure JPEG transmorphing[J].IET Signal Processing,2017,11(9):1031-1038.
[8]YANG H,XU X,XU C,et al.G2Face:High-Fidelity Reversible Face Anonymization via Generative and Geometric Priors[J].IEEE Transactions on Information Forensics and Security,2024,19:8773-8785.
[9]HUKKELÅS H,MESTER R,LINDSETH F.DeepPrivacy:AGenerative Adversarial Network for Face Anonymization[C]//Advances in Visual Computing:14th International Symposium on Visual Computing.Berlin:Springer-Verlag,2019:565-578.
[10]MAXIMOV M,ELEZI I,LEAL-TAIXÉ L.CIAGAN:Condi-tional Identity Anonymization Generative Adversarial Networks[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).2020:5446-5455.
[11]LI R,XIU Y,SAITO S,et al.Monocular Real-Time Volumetric Performance Capture[C]//European Conference on Computer Vision Cham:Springer,2020:49-67.
[12]KUANG Z,LIU H,YU J,et al.Effective De-identification Generative Adversarial Network for Face Anonymization[C]//Proceedings of the 29th ACM International Conference on Multimedia.New York:ACM,2021:3182-3191.
[13]LI J,HAN L,CHEN R,et al.Identity-Preserving Face Anonymization via Adaptively Facial Attributes Obfuscation[C]//Proceedings of the 29th ACM International Conference on Multimedia.New York:ACM,2021:3891-3899.
[14]GOODFELLOW I,SHLENS J,SZEGEDY C.Explaining andHarnessing Adversarial Examples[C]//2015 International Conference on Learning Representations(ICLR).2015.
[15]SZEGEDY C,ZAREMBA W,SUTSKEVER I,et al.Intriguing properties of neural networks[C]//2014 International Confe-rence on Learning Representations(ICLR).2014.
[16]SHAN S,WENGER E,ZHANG J,et al.Fawkes:ProtectingPrivacy against Unauthorized Deep Learning Models[C]//29th USENIX Security Symposium(USENIX Security 20).2020:1589-1604.
[17]ZHONG Y,DENG W.OPOM:Customized Invisible Cloak To-wards Face Privacy Protection[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2023,45(3):3590-3603.
[18]YANG X,DONG Y,PANG T,et al.Towards Face Encryption by Generating Adversarial Identity Masks[C]//2021 IEEE/CVF International Conference on Computer Vision(ICCV).IEEE,2021:3877-3887.
[19]DONG X,WANG R,LIANG S,et al.Face Encryption via Frequency-Restricted Identity-Agnostic Attacks[C]//Proceedings of the 31st ACM International Conference on Multimedia.New York:ACM,2023:579-588.
[20]TANG L,YE D,LYU Y,et al.Once and for All:UniversalTransferable Adversarial Perturbation against Deep Hashing-Based Facial Image Retrieval[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2024:5136-5144.
[21]GOODFELLOW I J,POUGET-ABADIE J,MIRZA M,et al.Generative adversarial nets[C]//Proceedings of the 27th International Conference on Neural Information Processing Systems.Cambridge,MA:MIT,2014:2672-2680.
[22]YIN B,WANG W,YAO T,et al.Adv-Makeup:A New Imperceptible and Transferable Attack on Face Recognition[C]//Twenty-Ninth International Joint Conference on Artificial Intelligence.2021:1252-1258.
[23]HU S,LIU X,ZHANG Y,et al.Protecting Facial Privacy:Gen-erating Adversarial Identity Masks via Style-robust Makeup Transfer[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).IEEE,2022:14994-15003.
[24]SHAMSHAD F,NASEER M,NANDAKUMAR K.CLIP2-Protect:Protecting Facial Privacy Using Text-Guided Makeup via Adversarial Latent Search[C]//2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).2023:20595-20605.
[25]JIA S,YIN B,YAO T,et al.Adv-attribute:inconspicuous andtransferable adversarial attack on face [C]//Proceedings of the 36th International Conference on Neural Information Processing Systems.Red Hook,NY:Curran Associates Inc.,2022:34136-34147.
[26]KARRAS T,LAINE S,AILA T.A Style-Based Generator Architecture for Generative Adversarial Networks[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).2019:4396-4405.
[27]LI Q,HU Y,LIU Y,et al.Discrete Point-Wise Attack is Not Enough:Generalized Manifold Adversarial Attack for Face Recognition[C]//2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).IEEE,2023:20575-20584.
[28]LIU J,LAU C P,CHELLAPPA R.DiffProtect:Generate Ad-versarial Examples with Diffusion Models for Facial Privacy Protection[J].arXiv:2305.13625,2023.
[29]PREECHAKUL K,CHATTHEE N,WIZADWONGSA S,et al.Diffusion Autoencoders:Toward a Meaningful and Decodable Representation[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).IEEE,2022:10609-10619.
[30]SONG J,MENG C,ERMON S.Denoising Diffusion ImplicitModels[C]//International Conference on Learning Representations.2020.
[31]SUN Y,YU L,XIE H,et al.DiffAM:Diffusion-Based Adversarial Makeup Transfer for Facial Privacy Protection[C]//2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).IEEE,2024:24584-24594.
[32]KIM G,KWON T,YE J C.DiffusionCLIP:Text-Guided Diffusion Models for Robust Image Manipulation[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).2022:2416-2425.
[33]SIMONYAN K,ZISSERMAN A.Very Deep ConvNets Net-works for Large-Scale Image Recogni;on[C]//2015 International Conference on Learning Representations(ICLR ).2015.
[34]PARK T,LIU M Y,WANG T C,et al.Semantic Image Synthesis With Spatially-Adaptive Normalization[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).2019:2332-2341.
[35]CHEN T,SCHMIDT M W.Fast Patch-based Style Transfer of Arbitrary Style[J].arXiv:1612.04337,2016.
[36]LI T,QIAN R,DONG C,et al.BeautyGAN:Instance-level Facial Makeup Transfer with Deep Generative Adversarial Network[C]//Proceedings of the 26th ACM International Confe-rence on Multimedia.ACM,2018:645-653.
[37]HUANG G B,MATTAR M,BERG T,et al.Labeled Faces in the Wild:A Database forStudying Face Recognition in Unconstrained Environments[C]//Workshop on Faces in “Real-Life” Images:Detection,Alignment,and Recognition.2008.
[38]LIU Z,LUO P,WANG X,et al.Deep Learning Face Attributes in the Wild[C]//2015 IEEE International Conference on Computer Vision(ICCV).2015:3730-3738.
[39]KARRAS T,AILA T,LAINE S,et al.Progressive Growing of GANs for Improved Quality,Stability,and Variation[C]//2018 International Conference on Learning Representations(ICLR).2018.
[40]GU Q,WANG G,CHIU M T,et al.LADN:Local Adversarial Disentangling Network for Facial Makeup and De-Makeup[C]//2019 IEEE/CVF International Conference on Computer Vision(ICCV).2019:10480-10489.
[41]HU J,SHEN L,SUN G.Squeeze-and-Excitation Networks[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.2018:7132-7141.
[42]DENG J,GUO J,XUE N,et al.ArcFace:Additive Angular Margin Loss for Deep Face Recognition[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).2019:4685-4694.
[43]SCHROFF F,KALENICHENKO D,PHILBIN J.FaceNet:A unified embedding for face recognition and clustering[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition(CVPR).2015:815-823.
[44]CHEN S,LIU Y,GAO X,et al.MobileFaceNets:Efficient CNNs for Accurate Real-Time Face Verification on Mobile Devices[C]//Chinese Conference on Biometric Recognition.Cham:Springer,2018:428-438.
[45]WANG Z,BOVIK A C,SHEIKH H R,et al.Image quality assessment:from error visibility to structural similarity[J].IEEE Transactions on Image Processing,2004,13(4):600-612.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市渝北区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发