Computer Science

Computer Science ›› 2025, Vol. 52 ›› Issue (6): 405-413.doi: 10.11896/jsjkx.241200001

• Information Security • Previous Articles    

Adversarial Face Privacy Protection Based on Makeup Style Patch Activation

YUAN Lin1, HUANG Ling1, HAO Kaile1, ZHANG Jiawei1, ZHU Mingrui2, WANG Nannan1,2, GAO Xinbo1   

  1. 1 Chongqing Key Laboratory of Image Cognition,Chongqing University of Posts and Telecommunications,Chongqing 400065,China
    2 State Key Laboratory of Integrated Services Networks,Xidian University,Xi'an 710071,China
  • Received:2024-12-02 Revised:2025-02-17 Online:2025-06-15 Published:2025-06-11
  • About author:YUAN Lin,born in 1989,Ph.D,asso-ciate professor.His main research in-terests include image and video proce-ssing,computer vision,multimedia security and privacy protection.
    GAO Xinbo,born in 1972,Ph.D,professor,Ph.D supervisor.His main research interests include artificial intelligence,machine learning,computer vision and pattern recognition.
  • Supported by:
    National Natural Science Foundation of China(62201107,U22A2096),Natural Science Foundation of Chongqing(CSTB2022NSCQ-MSX1265) and Science and Technology Research Program of Chongqing Municipal Education Commission(KJQN202300606).

PDF (PC)

90

Abstract: Facial recognition technology has developed rapidly,greatly facilitating people's lives,but it has also raised public concerns about personal privacy.Facial images shared by people through social media and the Internet may be collected by illegal organizations,which can use facial recognition systems to identify the identityand steal privacy information related to the users.Therefore,a privacy protection mechanism is needed to ensure that facial images published by users through public media can be viewed normally by people,but can prevent facial recognition systems from extracting accurate identity information.The mainstream adversarial sample-based methods can solve this problem to some extents,but they inevitably introduce noise that can be easily detected in the images.When people share personal photos on social media and other platforms,they often add some beauty effects.Therefore,embedding adversarial perturbations cleverly while adding beautification effects to the images to achieve identity privacy protection for the images is a win-win choice.In this regard,this paper proposes a facial image identity privacy protection method based on makeup style patch activation.This method activates the makeup style of the reference facial image into the features of the original facial image through feature patches,and then reconstructs the activated features into adversarial facial images with makeup.At the same time,it uses an identity privacy enhancement module to force the generated image's identity features to approach a target identity,thereby obtaining adversarial privacy protection capabilities.Experimental results show that the facial images generated by this method not only have better visual effects and a variety of makeup styles,but also can effectively defend against privacy infringement caused by various black-box facial recognition models.

Key words: Facial privacy, Makeup style, Feature patch, Identity privacy protection, Black-box face recognition model

CLC Number: 

  • TP751.1
[1]ZHANG S,FENG Y,BAUER L,et al.“Did you know this ca-mera tracks your mood?”:Understanding Privacy Expectations and Preferences in the Age of Video Analytics[C]//Proceedings on Privacy Enhancing Technologies.2021.
[2]WU Z,WANG Z,WANG Z,et al.Towards Privacy-Preserving Visual Recognition via Adversarial Training:A Pilot Study[C]//European Conference on Computer Vision.Cham:Springer,2018:627-645.
[3]MEDEN B,ROT P,TERHÖRST P,et al.Privacy-EnhancingFace Biometrics:A Comprehensive Survey[J].IEEE Transactions on Information Forensics and Security,2021,16:4147-4183.
[4]AGRAWAL P,NARAYANAN P J.Person De-Identification in Videos[J].IEEE Transactions on Circuits and Systems for Vi-deo Technology,2011,21(3):299-310.
[5]YUAN L,KORSHUNOV P,EBRAHIMI T.Secure JPEGscrambling enabling privacy in photo sharing[C]//2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition.2015:1-6.
[6]YUAN L,EBRAHIMI T.Image transmorphing with JPEG[C]//2015 IEEE International Conference on Image Processing(ICIP).2015:3956-3960.
[7]YUAN L,EBRAHIMI T.Image privacy protection with secure JPEG transmorphing[J].IET Signal Processing,2017,11(9):1031-1038.
[8]YANG H,XU X,XU C,et al.G2Face:High-Fidelity Reversible Face Anonymization via Generative and Geometric Priors[J].IEEE Transactions on Information Forensics and Security,2024,19:8773-8785.
[9]HUKKELÅS H,MESTER R,LINDSETH F.DeepPrivacy:AGenerative Adversarial Network for Face Anonymization[C]//Advances in Visual Computing:14th International Symposium on Visual Computing.Berlin:Springer-Verlag,2019:565-578.
[10]MAXIMOV M,ELEZI I,LEAL-TAIXÉ L.CIAGAN:Condi-tional Identity Anonymization Generative Adversarial Networks[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).2020:5446-5455.
[11]LI R,XIU Y,SAITO S,et al.Monocular Real-Time Volumetric Performance Capture[C]//European Conference on Computer Vision Cham:Springer,2020:49-67.
[12]KUANG Z,LIU H,YU J,et al.Effective De-identification Generative Adversarial Network for Face Anonymization[C]//Proceedings of the 29th ACM International Conference on Multimedia.New York:ACM,2021:3182-3191.
[13]LI J,HAN L,CHEN R,et al.Identity-Preserving Face Anonymization via Adaptively Facial Attributes Obfuscation[C]//Proceedings of the 29th ACM International Conference on Multimedia.New York:ACM,2021:3891-3899.
[14]GOODFELLOW I,SHLENS J,SZEGEDY C.Explaining andHarnessing Adversarial Examples[C]//2015 International Conference on Learning Representations(ICLR).2015.
[15]SZEGEDY C,ZAREMBA W,SUTSKEVER I,et al.Intriguing properties of neural networks[C]//2014 International Confe-rence on Learning Representations(ICLR).2014.
[16]SHAN S,WENGER E,ZHANG J,et al.Fawkes:ProtectingPrivacy against Unauthorized Deep Learning Models[C]//29th USENIX Security Symposium(USENIX Security 20).2020:1589-1604.
[17]ZHONG Y,DENG W.OPOM:Customized Invisible Cloak To-wards Face Privacy Protection[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2023,45(3):3590-3603.
[18]YANG X,DONG Y,PANG T,et al.Towards Face Encryption by Generating Adversarial Identity Masks[C]//2021 IEEE/CVF International Conference on Computer Vision(ICCV).IEEE,2021:3877-3887.
[19]DONG X,WANG R,LIANG S,et al.Face Encryption via Frequency-Restricted Identity-Agnostic Attacks[C]//Proceedings of the 31st ACM International Conference on Multimedia.New York:ACM,2023:579-588.
[20]TANG L,YE D,LYU Y,et al.Once and for All:UniversalTransferable Adversarial Perturbation against Deep Hashing-Based Facial Image Retrieval[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2024:5136-5144.
[21]GOODFELLOW I J,POUGET-ABADIE J,MIRZA M,et al.Generative adversarial nets[C]//Proceedings of the 27th International Conference on Neural Information Processing Systems.Cambridge,MA:MIT,2014:2672-2680.
[22]YIN B,WANG W,YAO T,et al.Adv-Makeup:A New Imperceptible and Transferable Attack on Face Recognition[C]//Twenty-Ninth International Joint Conference on Artificial Intelligence.2021:1252-1258.
[23]HU S,LIU X,ZHANG Y,et al.Protecting Facial Privacy:Gen-erating Adversarial Identity Masks via Style-robust Makeup Transfer[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).IEEE,2022:14994-15003.
[24]SHAMSHAD F,NASEER M,NANDAKUMAR K.CLIP2-Protect:Protecting Facial Privacy Using Text-Guided Makeup via Adversarial Latent Search[C]//2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).2023:20595-20605.
[25]JIA S,YIN B,YAO T,et al.Adv-attribute:inconspicuous andtransferable adversarial attack on face [C]//Proceedings of the 36th International Conference on Neural Information Processing Systems.Red Hook,NY:Curran Associates Inc.,2022:34136-34147.
[26]KARRAS T,LAINE S,AILA T.A Style-Based Generator Architecture for Generative Adversarial Networks[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).2019:4396-4405.
[27]LI Q,HU Y,LIU Y,et al.Discrete Point-Wise Attack is Not Enough:Generalized Manifold Adversarial Attack for Face Recognition[C]//2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).IEEE,2023:20575-20584.
[28]LIU J,LAU C P,CHELLAPPA R.DiffProtect:Generate Ad-versarial Examples with Diffusion Models for Facial Privacy Protection[J].arXiv:2305.13625,2023.
[29]PREECHAKUL K,CHATTHEE N,WIZADWONGSA S,et al.Diffusion Autoencoders:Toward a Meaningful and Decodable Representation[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).IEEE,2022:10609-10619.
[30]SONG J,MENG C,ERMON S.Denoising Diffusion ImplicitModels[C]//International Conference on Learning Representations.2020.
[31]SUN Y,YU L,XIE H,et al.DiffAM:Diffusion-Based Adversarial Makeup Transfer for Facial Privacy Protection[C]//2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).IEEE,2024:24584-24594.
[32]KIM G,KWON T,YE J C.DiffusionCLIP:Text-Guided Diffusion Models for Robust Image Manipulation[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).2022:2416-2425.
[33]SIMONYAN K,ZISSERMAN A.Very Deep ConvNets Net-works for Large-Scale Image Recogni;on[C]//2015 International Conference on Learning Representations(ICLR ).2015.
[34]PARK T,LIU M Y,WANG T C,et al.Semantic Image Synthesis With Spatially-Adaptive Normalization[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).2019:2332-2341.
[35]CHEN T,SCHMIDT M W.Fast Patch-based Style Transfer of Arbitrary Style[J].arXiv:1612.04337,2016.
[36]LI T,QIAN R,DONG C,et al.BeautyGAN:Instance-level Facial Makeup Transfer with Deep Generative Adversarial Network[C]//Proceedings of the 26th ACM International Confe-rence on Multimedia.ACM,2018:645-653.
[37]HUANG G B,MATTAR M,BERG T,et al.Labeled Faces in the Wild:A Database forStudying Face Recognition in Unconstrained Environments[C]//Workshop on Faces in “Real-Life” Images:Detection,Alignment,and Recognition.2008.
[38]LIU Z,LUO P,WANG X,et al.Deep Learning Face Attributes in the Wild[C]//2015 IEEE International Conference on Computer Vision(ICCV).2015:3730-3738.
[39]KARRAS T,AILA T,LAINE S,et al.Progressive Growing of GANs for Improved Quality,Stability,and Variation[C]//2018 International Conference on Learning Representations(ICLR).2018.
[40]GU Q,WANG G,CHIU M T,et al.LADN:Local Adversarial Disentangling Network for Facial Makeup and De-Makeup[C]//2019 IEEE/CVF International Conference on Computer Vision(ICCV).2019:10480-10489.
[41]HU J,SHEN L,SUN G.Squeeze-and-Excitation Networks[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.2018:7132-7141.
[42]DENG J,GUO J,XUE N,et al.ArcFace:Additive Angular Margin Loss for Deep Face Recognition[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).2019:4685-4694.
[43]SCHROFF F,KALENICHENKO D,PHILBIN J.FaceNet:A unified embedding for face recognition and clustering[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition(CVPR).2015:815-823.
[44]CHEN S,LIU Y,GAO X,et al.MobileFaceNets:Efficient CNNs for Accurate Real-Time Face Verification on Mobile Devices[C]//Chinese Conference on Biometric Recognition.Cham:Springer,2018:428-438.
[45]WANG Z,BOVIK A C,SHEIKH H R,et al.Image quality assessment:from error visibility to structural similarity[J].IEEE Transactions on Image Processing,2004,13(4):600-612.
[1] ZHENG Xu, HUANG Xiangjie, YANG Yang. Reversible Facial Privacy Protection Method Based on “Invisible Masks” [J]. Computer Science, 2025, 52(5): 384-391.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.